Study On Intelligent Optimization Algorithms And Its Application For Classification Problems

Intelligent optimization algorithms implement optimal functions by imitating biologic intelligent behavior. For example, genetic algorithm (GA) imitates the mechanism of natural selection in biologic population to solve optimal problem, and particle swarm optimization (PSO) imitates the coordinated mechanism of individual and swarm in migration of birds foraging to guiding the optimization search. Intelligent optimization algorithms have the characteristics of simple operations, generalization, robustness and parallelism, which have been extensively applied in various fields of pattern recognize, intelligent control, parallel search and associative memory.Support vector machines (SVM) are a kind of popular data classification technique. When SVM with Gaussian kernel are used in various fields of classification problems, first, some crucial problems to be confronted are: how to select the error penalty parameter C and the Gaussian kernel parameter?(namely how to conduct SVM model selection), how to optimize input feature subset so as to improve classification accuracy for SVM and reduce the feature subset. Parameter selection by user Enumeration method tends to reduce the classification performance of SVM and obtains very low classification accuracy, so that it can not meet the classification requirements. Grid search algorithm using SVM parameter optimization improves the accuracy of classification to a certain extent, initially to meet the classification requirements. As the requirements of data classification field to the classification accuracy are ever increasing, the use of the combination of intelligent optimization algorithm and SVM optimize the input feature subset and SVM parameters at the same time so as to further improve the classification accuracy. In the basis of above work, the asymptotic behaviors of SVM are fused with intelligent optimization algorithm. The genetic algorithm based on feature chromosomes, the particle swarm optimization algorithm based on feature particles and the clonal selection algorithm based on feature antibodies are proposed by this dissertation to construct the proposed algorithm and SVM hybrid system respectively to simultaneously optimize the feature subset and the SVM parameters, so as to obtain higher classification accuracy, smaller feature subset and shorter processing time. The basic methods of this dissertation are to construct hybrid systems of intelligent optimization algorithms and SVM to solve the above problems by fusing asymptotic behaviors of SVM into intelligent optimization algorithms.The topic of this dissertation is the study on intelligent optimization algorithms and its application for classification problems and the hybrid study of intelligent optimization algorithms and SVM. The contributions and innovations of this dissertation are summarized as follows.(1) Genetic algorithm based on feature chromosomesBased on the basic principles and search mechanism of genetic algorithm, the asymptotic behaviors of support vector machines are fused with genetic algorithm, which thereby directs the search of genetic algorithm to the straight line of optimal generalization error in the superparameter space by generating feature chromosomes operation. A genetic algorithm based on feature chromosomes, termed GAFC, is proposed to construct GAFC-SVM hybrid system so as to simultaneously optimize the feature subset and the parameters for SVM. The convergence of the proposed algorithm is analyzed. Compared with the similar approaches, the proposed approach not only has higher classification accuracy and smaller feature subsets, but also has fewer processing time.(2) Particle swarm optimization algorithm based on feature particlesBased on the basic principles and search mechanism of particle swarm optimization algorithm, the asymptotic behaviors of SVM are fused with particle swarm optimization algorithm by generating feature particles operation. A particle swarm optimization algorithm based on feature particles, called PSOFP, is proposed to construct PSOFP-SVM hybrid system so as to simultaneously optimize the feature subset and the parameters for SVM. The complexity of the proposed algorithm is analyzed. The experimental results indicate that the proposed algorithm has higher classification accuracy rates, smaller feature subsets and fewer processing time.(3) Clonal selection algorithm based on feature antibodiesBased on the basic principles and search mechanism of clonal selection algorithm, the asymptotic behaviors of SVM are fused with clonal selection algorithm by the generating feature antibodies operation. A clonal selection algorithm based on feature antibodies, termed CSAFA, is proposed to construct CSAFA-SVM hybrid system so as to simultaneously optimize the feature subset and the parameters for SVM. The experimental results indicate that the proposed algorithm has higher classification accuracy rate, smaller feature subset and better performance compared with the existing clonal selection algorithm and other classification ones.(4) Study on hybrid intelligent optimization algorithmsA hybrid algorithm of genetic algorithm based on feature chromosomes and quantum-inspired genetic algorithm, called GAFC-QGA hybrid algorithm, is proposed to construct GAFC-QGA-SVM hybrid system. The detailed experimental results are given to validate that the proposed algorithm is an effect method. In addition, a hybrid algorithm of clonal selection algorithm based on feature antibodies and differential evolution, termed CSAFA-DE hybrid algorithm, is proposed to construct CSAFA-DE-SVM hybrid system. The detailed experimental results are given to indicate that the proposed algorithm is a useful one.(5) Study on intelligent optimization algorithms applicationAccording the characteristics of microarray gene expression data, a hybrid system of wrapper method based on GAFC, filter method based on SNR and SVM, called GAFC-SNR-SVM hybrid system, is proposed so as to search the informative gene subset of small number of genes and high classification accuracy simultaneously. Experimental results indicate that the proposed algorithm has obvious advantages on the number of informative genes and classification accuracy rate. Moreover, a hybrid system of genetic algorithm based feature chromosomes, common spatial patterns and SVM, termed CSAFA-DE-SVM hybrid system, is proposed to perform classification parameter optimization of brain-computer interface, which obtains obvious effect in improving classification accuracy.