Swarm Intelligence Optimization Algorithm Based On Multiple Surrogate Models

There is a class of practical engineering optimization problems with the black-box characteristics.For instance,the mathematical representations of the objective functions are unknown,and the evaluation cost of the objective functions is high.The swarm intelligence optimization algorithm has been widely used for solving such problems because of the ability of handling the functions without continuity,convexity,derivability and feasible domain connectivity.However,the swarm intelligence optimization algorithm is a random search algorithm based on population,which needs a large number of fitness function evaluations to obtain a satisfactory result.Furthermore,the evaluation of the fitness function become extremely computationally expensive with the increase of the computational complexity in the engineering optimization problem.As a result,the whole optimization process of the swarm intelligence optimization algorithm may take a lot of computing resources.For reducing the number of fitness function evaluations while guaranteeing the satisfactory optimization result,the swarm intelligence optimization algorithm based on multiple surrogate models has been developed.With the help of the multi-surrogate optimization method,the swarm is guided to evolve,and its history information is employed to update the surrogate models.Then,the goals of less function evaluations and reasonable allocation of resources are achieved by the collaborative optimization.In this dissertation,the study on the swarm intelligence optimization algorithm based on multiple surrogate models is carried out in continuous optimization problem,mixed-integer optimization problem,multi-objective optimization problem and parallel computing.The main contents can be summarized as follows:1)A particle swarm optimization algorithm based on multiple surrogate models is proposed for the single objective function optimization problems in the continuous space,which consists of an inner loop optimization and an outer one.In the inner loop optimization,the search space is partitioned with a dynamic partitioning method,then the local surrogate models are constructed with the data in each subspace to take place of the global surrogate model for reducing modeling cost.Furthermore,a parallel optimization strategy based on multiple surrogate models is developed to achieve a balance between the global searching and local one.In outer loop optimization,the particle swarm optimization algorithm is guided by the results of the inner loop optimization,and its history information is used to update the training samples of the surrogate models.To verify the performance of the proposed algorithm,a number of numerical experiments are conducted by using ten benchmark test functions.The experimental results show that the proposed method can achieve satisfactory convergence results for low-dimensional,non-convex and multimodal problems.Then the proposed method is applied to the parameter determination problems of the data-driven model.The experimental results show that the proposed method is capable of determining the satisfied parameters on a limited computational buget.2)A discrete particle swarm optimization algorithm based on multiple surrogate models is proposed for the mixed-integer programming problems with constraints,in which the elitist particles are selected from the candidate solutions by the pre-selection strategy to guide the evolution of the particle swarm and the historical positions of the particle swarm are employed as the training samples of the surrogate model.To improve the efficiency of the pre-selection strategy,on one hand,a sampling method based on the multi-swarm collaboration model is developed to produce the candidate solutions with diversity.On the other hand,an improved Gaussian Process modeling method based on the data parallel approach is presented to construct the local surrogate model for reducing the modeling cost and improving the prediction ability in the local space.Finally,the effectiveness of the proposed method are verified by 12 benchmark test problems.Then the proposed method is applied to the hybrid optimization problem on the construction of samples and the determination of parameters.The experimental results show that the proposed method is feasible and effective.3)A multi-objective particle swarm optimization algorithm based on multiple surrogate models is proposed for the multi-objective optimization problems with high computational cost,which adopts the Pareto active learning method to classify the candidate solutions as the Pareto-optimal and non-Pareto-optimal ones.For achieving a balance between prediction accuracy and classification cost,an improved Pareto active learning classification strategy is proposed.Furthermore,to enhance the the quality of the candidate solutions,a mixed mutation sampling method based on the simulated evolution is presented.On the basis,the Pareto optimal solutions obtained are employed to guide PSO by updating the external archives,and the history information of PSO is used to update the surrogate model.Finally,a number of the benchmark test functions are employed to verify the performance of the proposed algorithm.Then the proposed method is applied to the parameter determination problems of the MIMO least squares support vector machine.The experimental results show that the proposed method is practical and effective in engineering.4)Based on the Bayesian Optimization and the heuristic technology,a parallel optimization algorithm based on adaptive surrogate model is proposed.Considering that the analytical solution of the multi-point expectation improvement criterion is difficult to be obtained,the heuristic method is employed to transform the auxiliary optimization problem based on the multi-point expectation improvement criterion into the one using the single-point expectation improvement criterion for reducing the computational complexity of the optimization.Furthermore,a data partitioning method is adopted to partition the input space.On the one hand,multiple local surrogate models are constructed,and then a single-point expectation improvement criterion based on the adaptive surrogate model is proposed to improve the accuracy of sampling.On the other hand,a hierarchical optimization method is presented,which is implemented in both the whole space and the local one to improve the efficiency of the optimizer.On the basis,the distributed parallel technology is adopted to calculate the real function values of query points for reducing the computational cost.Finally,the performance of the proposed method is analyzed from the aspects of the partition design,the parallel sampling and the experiment comparison.Furthermore,the superiority of the proposed method is verified by the parameter determination of the data-driven model.