Stochastic Optimization Algorithms Compare And Evaluate The Performance Of Quantitative Methods And Applications

Practical engineering optimization problems often have many complicated mathematical properties such as multiple-extremum, high nonlinearity, large-scale, discontinuity. Traditional optimization algorithms, especially gradient-based algorithms are difficult to get satisfactory solution in these situations. Recently, as a large category of optimization algorithms, stochastic optimization algorithm gradually plays an important role for its ability of global optimization, excellent adaptability as well as easy parallelization such as evolutionary computing, swarm intelligence optimization, simulated annealing, etc.Stochastic optimization algorithms have certain possibility to jump out of local optimum by introducing some random factors in optimization routine; however, random factors caused nonrepeatability of optimization results, which made difficulties for performance evaluation of stochastic optimization algorithms. Considering the randomness of the results, the relative performance superiority relationship between different stochastic optimization algorithms can only be probabilistic. Therefore, traditional methods of performance evaluation for deterministic optimization algorithms are no longer suitable for stochastic optimization algorithms.Currently, most methods evaluating performance of stochastic optimization algorithms are empirical and qualitative. Different methods may conduct totally different conclusions because of different metrics. In order to recognize difference of characteristic of different stochastic optimization algorithms correctly, method of performance evaluation for stochastic optimization algorithms must be systematic and unified as well as rigorous and quantitative. In order to meet these demands, a set of methods based on mathematical statistic theory is developed in this article, which can derive quantitative probabilistic indicators describing relative superiority about effectiveness and efficiency between different stochastic optimization algorithms. Further, multiple versions of particle swarm optimization (PSO) algorithm were utilized to implement this set of performance evaluation methods developed in this article as applications.For unconstrained stochastic optimization algorithms, some kind of random distribution was established as mathematical model of multiple optimization solutions of one optimization problem solved by some algorithm repeatedly under a certain computation cost. To determine the exact form of this distribution, statistic methods such as test of goodness of fit and maxium likelihood estimation were employed. With these distributions, probabilistic indicators describing relative superiority relationship about effectiveness and efficiency between two algorithms were difined and derived. As application, both the method developed in this article and traditional qualitative method based on history of mean value of multiple optimization results under equivalent computation cost were utilized to evaluate the relative performance superiority relationship between synchronous and asynchronous standard PSO algorithm. Results implied that the new method is not only consistent with traditional method metioned above but also helps analyzing detailed characteristics in optimizing process, which demonstrated the validity and significance of method developed in this article.For constrained stochastic optimization algorithms, a function describing the integrated quality of solutions including minimization degree of objective function value as well as feasibility was defined and derived. Using this function, the statistic qualitative methods of performance evaluation for unconstrained optimization algorithms can be easily generalized to constrained situation. As application, the relative performance superiority relationship between two versions of constraints-handling PSO algorithm employing penalty function and feasibility ranking strategy were evaluated. Results implied that quatitave conclusions were consistent with theoretical characteristics of these two versions.PSO algorithm is simple and highly efficient despite its relative weakness of global optimization capability. To maintain the diversity of particle swarm, a modified PSO version with mutation mechanism based on simulated annealing(SA) was developed. Numerical studies about key factors in SA mutation operator were carried out. Further, statistic quantitative performance evaluation between original version and the modified was implemented. Results proved that mutation mechanism indeed decreased the risk of trapping in local minima observably.This paper systematically studied the method of performance evaluation for both unconstrained and constrained stochastic optimization algorithms. Its conclusions help analyzing the detailed difference of characteristics between different stochastic optimization algorithms in the whole optimizing process as well as choosing high performance algorithm to solve specified practicle optimization problem. Moreover, PSO with SA mutation operator proved to be a simple but effective PSO version with global optimization capability. These studies were of some significance in theory and practice.