Research On Self-adaptive Distributed Differential Evolution Algorithm And Parallelization

With the rapid development of Internet technology and the explosive growth of data generated in various fields,large-scale data processing technology has become a hot topic in various research fields.Many optimization problems in real-world applications show largescale characteristics,and the field of data optimization has also entered a large-scale era.Because these optimization problems are highly complex and can not be effectively solved by traditional mathematical methods,evolutionary computation is a kind of efficient method for solving complex optimization problems.Although evolutionary algorithm is a kind of efficient method for solving complex optimization problems,this kind of method has changed the search space with exponential growth and the characteristics of search space because of the increase of the problem dimension.The performance of solution is reduced,and it is faced with the problem of "dimension disaster".Therefore,improving the accuracy and efficiency of evolutionary algorithm for large-scale optimization problems is a hot topic in academic and engineering applications.The distributed differential evolution algorithm,namely the differential evolution algorithm of structured population,is a highly competitive optimization method.In recent years,various improved versions have been used to solve large-scale optimization problems.In order to improve the accuracy and time efficiency of the distributed differential evolution algorithm for solving large-scale optimization problems,this paper studies the parallel distributed differential evolution algorithm based on the distributed computing framework Spark,and proposes an adaptive distributed differential evolution algorithm to improve the precision of the large-scale optimization problem.The main work of this article is as follows:(1)Based on the Spark platform,the DDE algorithm is parallelized,focusing on the realization of migration mechanisms based on two different mechanisms.Based on the experiment of cluster environment,the influence of the implementation of two different migration operators on the execution time of the algorithm is analyzed.At the same time,we also compare the acceleration ratios of two different implementations under different computing resources.The experimental results show that the migration operator based on Shuffle operation is more efficient and the speedup of algorithm increases with the increase of the number of cores.(2)In order to solve the problem that the fixed population resources can not be better used and rearranged in the distributed differential evolution algorithm,the DDE algorithm of dynamic adjustment strategy of subpopulation mutation operator is proposed in this paper.It can observe the subpopulation state through the learning stage,update the mutation operator of each subpopulation application at the next stage,and thus make it possible.In the whole evolutionary process,the fixed population resources use more performance mutation operators,which improves the precision of the distributed differential evolution algorithm for solving large-scale optimization problems.Based on the proposed parallel DDE method,this algorithm is implemented and tested on the CEC2013 for LSGO standard test function.The results show that,compared with the variant of the existing DDE algorithm,PDE and MPEDE,the optimization ability and convergence of DDE-SMS algorithm on most of the test functions is superior to the above two algorithms.