The Study Of Niching Particle Swarm Optimization Algorithms And Their Applications To Multiple Classifiers Ensemble

Particle Swarm Optimization (PSO) is an evolutionary computation technique based on swarm intelligence, which was inspired by social behavior of bird flocking or fish schooling. The niche technique was originated from genetic algorithm (GA) that can make a population-based stochastic optimization algorithm form a species so that multiple optimal solutions for a multimodal optimization problem can be found. Multiple classifiers ensemble (MCE) usually has better performance than a single classifier since they perform the classification by ensembling multiple classifiers. However, the MCE requires that each member (or base) classifier is of good performance and big diversity, which is similar to niche technique. Currently, some researchers found the relations between MCE and niche technique, and have applied existing niche techniques into MCE. These applications are still at developing stage because of the existing niche techniques having some intrinsic drawbacks.This dissertation briefly overviewed the development history, the current research status and the shortcomings of PSO and niche technique. After that, some improved ideas for niche technique were proposed. The main works in this dissertation can be introduced as follows:1. An explicit exploring information exchange mechanism was firstly proposed for niche techniques. Based on this idea, an adaptive sequential niche PSO algorithm was implemented. The proposed algorithm can avoid the inherent disadvantages of tranditional sequential niche algorithm. Particularly, this algorithm is independent of some parameters in tranditional sequential niche technique2. The exploring information exchange mechanism for niche PSO technique is further developed, and the necessity of the exploring information exchange mechanism for complicated multimodal optimization is further analyzed and investigated as well. Moreover, it was pointed out that the exploring information exchanges not only in sequential way but also in parallel way. In other words, the exploration information can be dynamically exchanged between the different individuals. In the light of this idea a multi-sub-swarm parallel niche PSO algorithm is implemented, which integrates a sequential technique with a parallel one. As a result, the advantage of the proposed method is that it has the running speed of the parallel technique, and also possesses the ability to share the search information effectively among the swarm like the sequential one.3. A novel ensemble multilayer pruning model was proposed for the MCE. A general pruning method for ensemble classifier can get only one optimal ensemble scheme. Under this circumstance, some member or base classifiers including useful information might be excluded in the pruning process. Nevertheless, in the multilayer ensemble pruning model each layer has multiple different selective ensemble, and they can take full advantage of the useful information owned by every base classifier. In this situation, each classifier will have opportunity to participate one ensemble.4. The proposed multiple-swarm niche parallel PSO algorithm was integrated with ensemble multilayer pruning classifier model. Finally, the ensemble multilayer pruning classifier model was performed in practice.