Study On Genetic Algorithms Based On MapReduce In The Big Data Environment

Genetic algorithms have advantages of strong robustness and strong versatility,and it solves problems without too much limitation.And it is considered to be one of the main technologies of big data processing.To cope with the challenge of the era of big data,this thesis carried out deeply research on genetic algorithms,and presented a processing problem of large scale and high solution quality based on cloud computing platform optimization genetic algorithm.Aiming at the defects of genetic algorithm premature convergence will be magnified in the big data environment,an improved genetic algorithm is proposed,which improved the algorithm's processing scale and also improved the solution quality of problems.In this thesis,firstly we study of the genetic algorithm based on the traditional platform.Taking feed formulation problems as the research model,we implement the standard genetic algorithm,analyze and summarize the experimental phenomena and results,and optimize the algorithm according to the characteristics of formulation problems.We propose a fast generation method of the initial population for the formulation problems,introduce the optimal preservation strategy,and put forward a new kind of crossover and mutation method with constraints.We study the relationship between the scale of the problem and the efficiency of the algorithm,the experiment found that even if the algorithm is optimized very well,quality has been significantly improved,when the scale of the problem to a certain extent,the efficiency of the algorithm has been unable to meet the demand,it cannot adapt to the environment of big data.To solve this problem,in this thesis the genetic algorithm is transplanted into cloud computing platform for research to meet the challenge of big data.This thesis gives the flow of the standard genetic algorithm in MapReduce model,compare and analyze the standard genetic algorithm in Hadoop platform with the genetic algorithm in traditional platform respectively from the solution quality and the scale of processing problem.Experiments find that the migration of genetic algorithms to cloud computing platform can improve the problem's size that the algorithm can deal with,but if do not take some optimizations of the algorithm,the quality of the solution can't be improved either.Next,aiming at the low quality solution obtained by genetic algorithms and its weak global search capability based on cloud computing platform,this thesis implements some optimizations on the algorithm combined with the characteristics of genetic algorithms and cloud computing platform,considering the characteristics of large size data problems.Adding the sharing mechanism of niche technology to the selection operation and adding the pre-selection mechanism to the crossover and mutation operation,improved the global search ability of the algorithm and had maintained the population's diversity.Finally we have carried on the comparison and analysis on the algorithm's efficiency,and verified the optimized algorithm based on MapReduce has feasibility and effectiveness in dealing with large-scale problems.