Parameters Optimization Of SVMs Based On Memetic Algorithms

Support Vector Machines (SVMs) are a powerful machine learning method and havebeen successfully applied to model the data sets with few samples and high dimensions forclassification and regression. A critical issue concerned with the performance of SVMsmodels is how to determine the kernel parameters and the hyper parameters.This study mainly concentrates on the parameters optimization of SVMs, which hasgained many attentions in the existing literatures. A simple and direct way is to use anexhaustive Grid Search in the parameter space, which is time consuming, especially in thecase of regression problems with more than two parameters to be optimized. Numericaloptimization methods by minimizing the generalization error bounds are generallyefficient with fast convergence rate but subject to the high sensitivity to starting points.Consequently, they are liable to get stuck in local optimal. Evolutionary Algorithms (EA)are widely used for parameter optimization in SVMs for their global search ability recently.But this kind of methods usually suffers from premature convergence and lacks localsearch ability in promising regions.In this study, by combining Particle Swarm Optimization algorithm (PSO) andPattern Search (PS), an efficient memetic algorithm is proposed to optimize theparameters of SVMs. In the proposed MAs, PSO is responsible for exploration of thesearch space and the detection of the most potential regions with optimum solution, whilea direct search algorithm, pattern search (PS) in this case, is used to produce an effectiverefinement on the most potential regions obtained by PSO. For the purpose of validation,three experiments are conducted in this study. The first experiment aims to examine howthe pattern search can enhances the local search ability of PSO in the proposed MAs. Theperformance of proposed MAs for parameter optimization in SVMs is justified in the resttwo experiments for regression and classification problems respectively. The results, compared with some established counterparts, show the proposed approach can yieldpromising results for parameter optimization in SVMs both in regression and classificationproblems.