Research And Application Of A Hybrid Dynamic Multi-swarm PSO Algorithm With Nelder-Mead Simplex Method

Particle Swarm Optimization (PSO) algorithm proposed by Kennedy andEberhart in1995, is a typical swarm intelligent algorithm which imitates the groupbehaviour of birds. Since it was easy to be computed with fast convergence speed,good robustness and a few parameters to be tuned, PSO algorithm gained extensiveattention of domestic and overseas scholars in many research fields. However, PSOhas some apparent drawbacks: firstly, it isn’t a theoretically global convergentalgorithm; secondely, PSO often fails to find accurate soutions because it lacks ofcooperation from other precise search methods; finally, it is a heuristic bionicoptimization algorithm that is poorly supported by rigorous mathematical theories.With these in mind, many researchers were dedicated to improve the original PSO andenormous of new variants were available in recent yeas. Among them, the DynamicMulti-Swarm Particle Swarm Optimizer (DMS-PSO) proposed by Liang et al. is awidely recognized excellent algorithm which has powerful global search ability butpoor at local search. To improve its local search, Liang et al. proposed a variantversion of DMS-PSO, i.e. DMS-L-PSO which combines DMS-PSO algorithm with aclassical local search algorithm--Quasi-Newton Method. However, Quasi-Newtonmethod requires calculating the first derivatives of the function.(Though they may beapproximated with finite differences, it would be complex). In some cases where wedo not have any knowledge about the function to be optimized, DMS-L-PSO wouldn’tbe applicable. Whereas, Nelder-Mead (NM) simplex method is a famous simple directsearch technique that does not require any gradient information. This paper tries tointegrate NM simplex method into DMS-PSO, and proposes a hybrid dynamicmulti-swarm PSO algorithm with Nelder-Mead simplex method (DMS-NM-PSO) todeal with optimization problems where objective function is not continuous or notderivative.This paper studies the fundamentals, flow, parameter selection and practicalapplications of DMS-NM-PSO. Specifically, a sensitivity analysis experiment isdesigned to investigate parameter selection of our proposed algorithm. In a suite of25test functions taken from CEC05, DMS-NM-PSO is compared with DMS-PSO andDMS-L-PSO by using independent t-test, respectively. The experimental results showthat DMS-NM-PSO achieves better optimization effect than DMS-PSO does showingthat the introduce of NM simplex method can indeed improve the original algorithm,and that our proposed algorithm is also competitive compared to DMS-L-PSO. Inaddition, a Wilcoxon signed ranks nonparametric test of obtained results is carried outto provide a complete performance assessment of DMS-NM-PSO with other13famous algorithms, which demonstrates that our proposed algorithm shows asignificant improvement over, or at least statistically equals to almost all of thereferenced algorithms.The application researches on DMS-NM-PSO include:(1) optimizing initialweights and biases of BP neural networks to approximate nonlinear functions, and in a simulation experiment approximating6functions DMS-NM-PSO is superior toGenetic Algorithm (GA) and PSO algorithms;(2) optimizing two control parametersin Support Vector Machine (SVM) to automaticly classify musical genres. In thisapplication, we adopt real number encoding and use cross validation accuracy as thefitness value. It is also proved by the simulation results that DMS-NM-PSO performsbetter than GA and PSO.In the end of this paper, we summarize our main findings and discuss the futureworks and other application fields of DMS-NM-PSO.