IRON KNIGHTS METAL WORKING

The Overview of Genetic Algorithms and Evaluation Process for the Solution of Optimization Problems

I.  Introduction

The development and use of optimization models is well established. However, the use of many models has been restricted in some fields of economic analysis where the problem is large in size and there are a large number of non-linear interactions. In most cases, the use of linear approximations or simplification of the model has been necessary in order to derive a solution. GA follows the concept of solution evolution by stochastically developing generations of solution populations using a given fitness statistic. They are particularly applicable to problems which are large, non-linear and possibly discrete in nature. Genetic algorithms make it possible to explore a far greater range of potential solutions to a problem than do

conventional programs. It is designed to simulate processes in natural system necessary for evolution, specifically those that follow the principles of survival of the fittest [1]. As such they represent an intelligent exploitation of a random search within a defined search space to solve a problem. The genetic algorithm's implicit parallelism allows it to test and exploit large numbers of regions in the search space while manipulating relatively few strings.

            II. GENETIC ALGORITHMS

Genetic algorithms are a probabilistic search approach which is founded on the ideas of evolutionary processes. The GA procedure is based on the Darwinian principle of survival of the fittest. An initial population is created containing a predefined number of individuals (or solutions), each represented by a genetic string (incorporating the variable information). Each individual has an associated fitness measure, typically representing an objective value. The concept that fittest (or best) individuals in a population will produce fitter offspring is then implemented in order to reproduce the next population. Selected individuals are chosen for reproduction (or crossover) at each generation, with an appropriate mutation factor to randomly modify the genes of an individual, in order to develop the new population. The result is another set of individuals based on the original subjects leading to subsequent populations with better (min. or max.) individual fitness. Therefore, the algorithm identifies the individuals with the optimizing fitness values, and those with lower fitness will naturally get discarded from the population.

Ultimately this search procedure finds a set of variables that optimizes the fitness of an individual and/or of the whole population [2]. As a result, the GA technique has advantages over traditional non-linear solution techniques that cannot always achieve an optimal solution. A simplified comparison of the GA and the traditional solution techniques is illustrated in Figure 1. Non-linear programming solvers generally use some form of gradient search technique to move along the steepest gradient until the highest point (maximization) is reached. In the case of linear programming, a global optimum will always be attained. However, non-linear programming models may be subject to problems of convergence to local optima, or in some cases, may be unable to find a feasible solution. This largely depends on the starting point of the solver. A starting point outside the feasible region may result in no feasible solution being found, even though feasible solutions may exist [3]. Other starting points may lead to an optimal solution, but it is not possible to determine if it is a local or global optimum. Hence, the modeler can never be sure that the optimal solution produced using the model is the “true” optimum.

 

 

Fig.1. Gradient search technique

 

 

 

 

 

Fig. 1(a) A genetic algorithm approach

For the genetic algorithm, the population encompasses a range of possible outcomes. Solutions are identified purely on a fitness level, and therefore local optima are not distinguished from other equally fit individuals. Those solutions closer to the global optimum will thus have higher fitness values. Successive generations improve the fitness of individuals in the population until the optimization convergence criterion is met. Due to this probabilistic nature GA tends to the global optimum, however for the same reasons GA models cannot guarantee finding the optimal solution as shown in the Fig.1 (a).

The GA consists of four main stages: evaluation, selection, crossover and mutation. The evaluation procedure measures the fitness of each individual solution in the population and assigns it a relative value based on the defining optimization (or search) criteria. Typically in a non-linear programming scenario, this measure will reflect the objective value of the given model. The selection procedure randomly selects individuals of the current population for development of the next generation. Various alternative methods have been proposed but all follow the idea that the fittest have a greater chance of survival. The crossover procedure takes two selected individuals and combines them about a crossover point thereby creating two new individuals. Simple (asexual) reproduction can also occur which replicates a single individual into the new population. The mutation procedure randomly modifies the genes of an individual subject to a small mutation factor, introducing further randomness into the population.

This iterative process continues until one of the possible termination criteria is met: if a known optimal or acceptable solution level is attained; or if a maximum number of generations have been performed; or if a given number of generations without fitness improvement occur [4]. Generally, the last of these criteria applies as convergence slows to the optimal solution.

Population size selection is probably the most important parameter, reflecting the size and complexity of the problem. However, the trade-off between extra computational efforts with respect to increased population size is a problem specific decision to be ascertained by the modeler, as doubling the population size will approximately double the solution time for the same number of generations. Other parameters include the maximum number of generations to be performed, a crossover probability, a mutation probability, a selection method and possibly an elitist strategy, where the best is retained in the next generation’s population.

Unlike traditional optimization methods, GA is better at handling integer variables than continuous variables. This is due to the inherent granularity of variable gene strings within the GA model structure. Typically, a variable is implemented with a range of possible values with a binary string indicating the number of such values; i.e. if x1 Î [0, 15] and the gene string is 4 characters (e.g. “1010”) then there are 16 possibilities for the search to consider. To model this as a continuous variable increases the number of possible values significantly [5]. Similarly, other variable information which aids the search considerably is upper and lower bound values. These factors can affect convergence of the model solutions greatly.

            III. TYPICAL EVALUATION PROCESS IN GA FOR OPTIMIZATION

Generally, GA has the following elements: populations of chromosomes, selection according to fitness, crossover to produce new offspring, and random mutation of new offspring.  Each chromosome can be a point in the search space of candidate solutions.  The GA processes populations of chromosomes, successively replacing one such population with another.  The GA most often requires an objective function that assigns a fitness score to each chromosome in the current population.  The fitness of a chromosome depends on how well that chromosome solves the problem at hand. The following Figure. 2 illustrates the typical evolution process of the GA.

Fig.2 Typical evaluation process of GA

The key issue is that evolution is a massively parallel (global) search method.  Unlike the iterative local searching approach, such as Snake, which has only one start point and converges to only one result, a population of chromosomes is maintained in GA during the whole period of the evolution and could converge to more than one peak in the solution space simultaneously[6,7].

In our implementation, the chromosome is defined as the 9 weights and some additional parameters that will help to reconstruct a boundary from the 9 weights.   Therefore, each chromosome stands for a potential boundary that may be the solution of our problem.  It can be observed that the prostate appears in an ultrasound image as a relatively dark region.  Thus, the objective function is defined so that high fitness score is given to those potential boundaries that has darker region inside and brighter region outside.  Those boundaries, e.g., chromosomes, will have a chance to mate and produce the next generation of chromosomes.

 

IV. GENETIC ELEMENTS OF GENETIC ALGORITHMS

The algorithms must have these elements to be "genetic"

v      A mathematical representation for the solutions. This is a string of values. A value in a certain position has a special meaning (just like our genes do). Keeping the space optimization problem in mind, we could have a string representing how to position 3 boxes in x, y, z coordinates. For simplicity, each coordinate is represented by a decimal number, so position 1 tells us the x coordinate of box 1, position 2 is the y coordinates of box 1, and position 3 is the z coordinates of box 1. Position 4 would then be the x coordinates of box 2, etc. It could look something like this (the 9 digit string is the "gene string" representing a specific solution)

v      A method of creating the initial population. You determine how many individuals you want. If you have an idea of the best solution you can initialize the individuals accordingly. However, most of the time you are probably better off by sampling starting with random values.

v      A method for measuring the fitness. You have to have a measure of fitness in order to select the best individuals. In this example the obvious measure is how much free space the proposed solution will offer. The more free space, the better the solution is.

v      Genetic functions. These are the selection, cross-over, and mutation methods mentioned before. I will explain them below; for now it is enough to say that they recombine existing individuals into new individuals, representing new solutions.

A number of parameters. You have to determine in advance the size of the population, the number of parents to select, the mutation rate, etc.

This flowchart illustrates the basic steps in a GA as shown in the Fig.3.

 

  Initialize

Population

Population

Calculate

  Fitness

 Transfer

Solution

 Found?

Offspring

   Genetic

Operations

    Stop

Iterations

 

Fig.3. Basic steps in GA

V.  Applications of Genetic Algorithms


(A). GA on optimization and planning: Traveling Salesman Problem (TSP)

The TSP is interesting not only from a theoretical point of view, many practical applications can be modeled as a traveling salesman problem or as variants of it, for example, pen movement of a plotter, drilling of printed circuit boards (PCB), real-world routing of school buses, airlines, delivery trucks and postal carriers. Researchers have tracked TSPs to study bimolecular pathways, to route a computer networks' parallel processing, to advance cryptography, to determine the order of thousands of exposures needed in X-ray crystallography and to determine routes searching for forest fires (which is a multiple-salesman problem partitioned into single TSPs). Therefore, there is a tremendous need for algorithms. In the last two decades an enormous progress has been made with respect to solving traveling salesman problems to optimality which, of course, is the ultimate goal of every researcher. One of landmarks in the search for optimal solutions is a 3038-city problem. This progress is only party due to the increasing hardware power of computers [8]. Above all, it was made possible by the development of mathematical theory and of efficient algorithms. There are strong relations between the constraints of the problem, the representation adopted and the genetic operators that can be used with it. The goal of traveling Salesman Problem is to devise a travel plan (a tour) which minimizes the total distance traveled. TSP is NP-hard (NP stands for non-deterministic polynomial time) - it is generally believed cannot be solved (exactly) in time polynomial. The TSP is constrained:

v      The salesman can only be in a city at any time

v      Cities have to be visited once and only once.

When Genetic Algorithms applied to very large problems, they fail in two aspects: They scale rather poorly (in terms of time complexity) as the number of cities increases. The solution quality degrades rapidly. To use a standard GA the following problems have to be solved:

v      A binary representation for tours is found such that it can be easily translated into a chromosome.

v      An appropriate fitness function is designed, taking the constraints into account.

 

v      Non-permutation matrices represent unrealistic solutions, that is, the GA can generate some chromosomes that do not represent valid solutions. This happens: in the random initialization step of the GA. as a result of genetic operators (mutation and crossover). Thus, permutation matrices are used. Two tours including the same cities in the same order but with different starting points or different directions are represented by different matrices and hence by different chromosomes, for example:

Tour (23541) = tour (12354)

(i) Evolutionary Divide and Conquer (EDAC)

This approach, EDAC has potential for any search problem in which knowledge of good solutions for sub problems can be exploited to improve the solution of the problem itself. The idea is to use the Genetic Algorithm to explore the space of problem subdivisions rather than the space of solutions themselves, and thus capitalize on the near linear scaling qualities generally inherent in the divide and conquer approach. Moreover, EDAC approach is intrinsically parallel. The EDAC approach has lifted the application of GA to TSP an order or magnitude larger in terms of problem sizes than permutation representations. Experimental results demonstrate the successful properties for EDAC on uniform random points and PCB problems in the range 500 - 5000 cities.

(B). GA in Business and Their Supportive Role in Decision Making

Genetic Algorithms have been used to solve many different types of business problems in functional areas such as finance, marketing, information systems and production/operations. Within these functional areas, Genetic Algorithms has performed a variety of applications such as tactical asset allocation, job scheduling, machine-part grouping, and computer network design.

(i) Finance Applications

Models for tactical asset allocation and international equity strategies have been improved with the use of Genetic Algorithms. They report an 82% improvement in cumulative portfolio value over a passive benchmark model and a 48% improvement over a non-GA model designed to improve over the passive benchmark. Genetic algorithms are particularly well-suited for financial modeling applications.

(ii) Production/Operation Applications

Genetic Algorithm has been used to schedule jobs in a sequence dependent setup environment for a minimal total tardiness. All jobs are scheduled on a single machine; each job has a processing time and a due date. The setup time of each job is dependent upon the job which immediately precedes it. The GA is able to find good, but not necessarily optimal schedules, fairly quickly. It is also used to schedule jobs in non-sequence dependent setup environment. The jobs are scheduled on one machine with the objective of minimizing the total generally weighted penalty for earliness or tardiness from the jobs' due dates. However, this does not guarantee that it will generate optimal solutions for all schedules. It is developed for solving the machine-component grouping problem required for cellular manufacturing systems. It provides a collection of satisfactory solutions for a two objective environment, allowing the decision maker to then select the best alternative.

(iii)- Role in Decision Making

Applying the well established decision processing phase model of Simon (1960), Genetic Algorithms appear to be very well suited for supporting the design and choice phases of decision making. In solving a single objective problem, GA designs many solutions until no further improvement (no increase in fitness) can be achieved or some predetermined number of generations has evolved or when the allotted processing time is complete. The fit solution in the final generation is the one that maximizes or minimizes the objective (fitness) function; this solution can be thought as it has recommended choice. Therefore with single objective problems the user of GA is assisted in the choice phase of decision processing. When solving multi-objective problems, it gives out many satisfactory solutions in terms of the objectives, and then allows the decision maker to select the best alternative. Therefore it assists with the design phase of decision processing with multi-objective problems. Its  can be of great assistance for examining alternatives since they are designed to evaluate existing potential solutions as well to generate new (and better) solutions for evaluation. Thus it can improve the quality of decision making.

(C). Learning Robot behavior using Genetic Algorithms

Robot has become such as prominent tools that it has increasingly taken a more important role in many different industries. As such, it has to operate with great efficiency and accuracy. This may not sound very difficult if the environment in which the robot operates remain unchanged, since the behaviors of the robot could be pre-programmed. However, if the environment is ever-changing, it gets extremely difficult, if not impossible, for programmers to figure out every possible behaviors of the robot. Applying robot in a changing environment is not only inevitable in modern technology, but is also becoming more frequent. This has obviously led to the development of a learning robot.  

The approach to learning behaviors, which lead the robot to its goal, described here reflects a particular methodology for learning via simulation model. The motivation is that making mistakes on real system can be costly and dangerous. In addition, time constraints may limit the extent of learning in real world. Since learning require experimenting with behaviors that might occasionally produce undesirable results if applied to real world. Therefore, as shown in the diagram, the current best behavior can be place in the real, on-line system, while learning continues in the off-line system.

(D). Advantages and Disadvantages of Genetic Algorithms

A GA has a number of advantages. It can quickly scan a vast solution set. Bad proposals do not affect the end solution negatively as they are simply discarded. The inductive nature of GA means that it doesn't have to know any rules of the problem - it works by its own internal rules. This is very useful for complex or loosely defined problems. , the parallel nature of its stochastic search is one of the main strengths of the genetic approach GA has drawbacks too, of course.

While the great advantage of GA is the fact that they find a solution through evolution, this is also the biggest disadvantage. Evolution is inductive; in nature life does not evolve towards a good solution - it evolves away from bad circumstances. This can cause a species to evolve into an evolutionary dead end. GA is usually slower than traditional techniques.

Consider this example: a GA must find the highest point in a landscape. The algorithm will favor solutions that lie on a peak (a hill or whatever). As the individuals gradually begin to propose solutions in the same vicinity (somewhere on the peak), they begin to look alike. In the end you may have individuals that are almost identical. The best of these suggest the top of the peak as the solution. But what if there is another, higher peak at the other end of our map? It is too hard for the individuals to venture away from their current peak. The ones that do will be eliminated, because their solution is worse than the ones we have. An individual might get "lucky", but that would mean its "genes" are very different from the rest of the population, so this is unlikely. In other words, the algorithm produces a suboptimal solution - and we may not even know it.

 

VI. CONCLUSION

GA is an evolutionary approach which is an alternative to traditional optimization methods. It is most appropriate for complex, non-linear model where location of the global optimum is a difficult task. It may be possible to use some techniques to consider problems which may not be modeled as accurately. Genetic approaches are often simple to design and easy to code, and can be used to greatly increase the probability of finding the true global optimum of multi-dimensional functions. We argue that a careful analysis of developmental mechanisms is useful when understanding the success of several standard techniques, and can clarify the relationships between more recently proposed enhancements.

 The applications, be they commercial, educational and scientific, are increasingly dependent on this algorithms, the Genetic Algorithms. Its usefulness and gracefulness of solving problems has made it the more favorite choice among the traditional methods, namely gradient search, random search and others. It is very helpful when the developer does not have precise domain expertise, because it possesses the ability to explore and learn from their domain. In future, we would witness some developments of variants of GA to tailor for some very specific tasks. Generally, a standard genetic algorithm is taken for specific development of the problem under investigation where the modeler should take advantage of model structure for effective implementation. There are a number of factors which must be taken into consideration when developing a genetic algorithm model; there are typically many standard parameters which can be modified to affect the performance of the optimization, variable specification (probabilistic or deterministic), tight variable bounds, weighting strategies and constraints Lastly, the success and ease of implementation of a model within a GA is directly related to the knowledge of the modeler on three topics; programming, model understanding and basic knowledge of GA. It is beneficial spending time to construct an efficient model with (few) control variables in a largely feasible variable space.

VII. References

 

[1]. Rajasekaran, Vijayalakshmi, 2005    S. Rajasekaran,        G.A.Vijayalakshmi, Neural networks, fuzzy logic & genetic              algorithms synthesis and applications, Prentice Hall  of India        Pvt. ltd, 2005 (5th edition).

 

[2]. Patterson, 2003 Dan W. Patterson, Introduction to AI and expert        system, Prentice Hall of India Pvt.Ltd., New Delhi, 2003.

 

[3]. Goldberg, 1989 Goldberg D.E., Genetic algorithms in search,        optimization and machine learning, Addision-wesley, USA,        1989.

 

[4]. Louis,1993Sushil J. Louis, Genetic Algorithms as a        Computational Tool for Design, August 1993.

 

[5]. Brooke,1998 Brooke, A.D., D. Kendrick and A. Meerhaus,        GAMS: A User’s Guide, Scientific Press, 1988.

 

[6]. Carroll, 1997 Carroll, D.L., Fortran GA - Genetic Algorithm        Driver V1.6.4, Users Guide, 1997.

 

[7]. Baeck,1998 Baeck, T. A User’s Guide to GENEsYs 1.0,        Department of Computer Science, University of Dortmund, 1998.

       G.Winter, J.Periaux & M.Galan, Genetic Algorithms in         Computer Engineering.      

 

[8]. Darrell Whitley , Vose, 1995 L.Darrell Whitley & Michael        D.Vose, Foundatiions of Genetic Algorithms Volume 3,

       Morgan Kaufmann Publishers, Inc.,1995.

 

 

About the Author

Assistant professor in lord venkateswara engineering college.I am doing phd in sathyabama university, Tamil Nadu,India.shankar_submanian@yahoo.co.in

A Tech Authority - HEIDENHAIN LS 186 Linear Scale