Study Of Several Optimization Algorithms For Support Vector Machine

Support Vector Machine (SVM) is a new machine learning method and has a rapid development in recent years, which has been successfully applied into the fields of data classification and regression such as handwritten digit recognition, face recognition, text classification, regression prediction and time serial analysis etc. How to design the efficient optimization algorithms for solving SVM has become an open problem in academic research.In this paper, the model parameter selection of support vector machine for classification and regression are studied, the influences of the kernel function and the related parameter on the generalization ability of support vector machine are analyzed. Based on these discussions, an algorithm to search the best suitable parameters with grid-search and k-fold cross-validation method is introduced. On the basis of theoretical analysis, the numerical experiments achieve good results.Incremental learning has being a research focus in recent years, its advantage is that the learning process can automatically discard useless samples, reduce the training set and save storage costs. Though classic or standard SVM algorithm does not have incremental ability to learn, its theoretical system of support vector concept to incremental learning algorithm construction is great significant. The improved SOR-SVM is an effective algorithm to solve the SVM problem. In this paper, the improved SOR-SVM algorithm is extended to the incremental learning, an online incremental algorithm based on improved SOR-SVM and bulk incremental algorithm based on improved SOR-SVM are proposed. By analyzing the KKT conditions and using the algorithm's special structure, the result which has been obtained before the incremental process can be used in the solving process with the new additional samples, thus the amount of computation is significantly reduced and the efficiency is greatly improved. Numerical experiments show that the new incremental learning algorithm is useful.Similar with the classification problem, in this paper, the improved SOR-SVR algorithm is extended to the incremental learning, an online incremental algorithm based on improved SOR-SVR and bulk incremental algorithm based on improved SOR-SVR are proposed. By analyzing the KKT conditions and using the algorithm's special structure, solving process with the new additional samples can use the result which has been obtained before the incremental process, thus significantly reduces the amount of computation and greatly improves the efficiency. Numerical experiments show that the new incremental learning algorithm is useful.Genetic Algorithm is a parallel, random and adaptive searching algorithm, which bases on the mechanics of natural selection and natural genetics. In this paper, a support vector regression algorithm based on genetic algorithm is proposed, the algorithm is different from the traditional method of quadratic programming. Numerical experiment results have demonstrated that the algorithm is a powerful tool for solving regression problems and has good robustness and generalization ability.