Study On Application Of Machine Learning Based On Support Vector Machine

During the last decades, the method of Support Vector Machine (SVM) based on the Statistical Learning Theory became an important research field in machine learning. Different from those traditional algorithms based on empirical risk minimization rule, SVM is based on structural risk minimization rule. So SVM can achieve a good balance between empirical risk and classifier capacity. In addition, SVM has other advantages such as global optimization, excellent adaptability and generalization ability. However, there are still some problems with SVM, such as too time-consuming and difficulty of selecting kernel parameters, which restricts the application of SVM. Thus, our study focuse on the above- mentioned issues. The research results have been tested on several benchmark data sets of the world.The contributions of this dissertation include:1) Study on the training algorithm of SVM. Currently, sequential minimal optimization (SMO) algorithm has become the best training algorithm for SVM, Working set selection is the key of implementing SMO. We studied the working set selection strategy based on Zoutendijk's maximal descent direction method and function approximation deeply, and deduced it strictly. We found that the existing selection method has some defaults when the Hessian matrix of the quadric programming problem is not positive definite.2) Study on the reduction for large-scale training set. SVM has better performance than other learning algorithms in case of small sample. But that does not mean that SVM is only used for small sample. As a matter of fact, a majority of problems encountered in real world belong to large-scale data set. In case of large sample, even the SMO algorithm consumes too much training time and can not satisfy the real-time requirements. Based on the geometry distribution of the support vectors, two reduction strategies for pre-selection of support vectors in primal input and high-dimentional feature space are proposed. Enlightened by clustering method, the strategies for for pre-selection of support vectors in high-dimentional space is based on category centroid of sample. Enlightened by a combination of the nearest neighbor rule and geometric distribution, the strategies for for pre-selection of support vectors in primal input space is based on the approach of searching boundary set including support vector using Delaunay Triangulations network. Experiments results show that these two reduction strategies are effective in that they can reduce training time sharply without downgrading the generation ability and prediction accuray.3) Study on model selection of SVM. After mapping the samples from primal input space to high-dimentional feature space (Hilbert space) using kernel function, we can obtain linear discriminant hyperplane in the feature space. Different kernels correspond to different feature spaces, and different results of SVMs are obtained by mapping based on different kernels. By measuring the linear discrimination degree and the complexity of models, a feature space with good generation ability for learning machine is found, and kernel selection is performed. Once feature space is determined, the relationship between the penalty factor and the margin is analyzed, and penalty factor is selected by means of the margin. Instead of building an analytic formula reflecting the relationship between kernel function, penalty factor and generalization ability, the proposed model selection method seeks to estimate the effect of parameter selection on generalization performance indirectly and provide a guide for model selection.4) Study on application of SVM. In this paper, two typical problems are studied using SVM in detail. Firstly, for the typical problem of pattern classification - face recognition, a new identification method based on face component is proposed. Because one-against-rest SVM classifation algorithm is still lack of theoretic foundation, another multi-classification algorithm with similarity as discrimination standard, which use posterior probability as output of SVM is proposed. The experiment is conducted on the ORL and YALE face image database. The result indicates that the proposed method is robust in that it is insensitive to expression and pose variations. Secondly, a classical application in financial domain, personal credit evaluation, is studied. In particular, the application of two feature selection and extraction methods based on SVM (genetic algorithm and principal component analysis) is discussed. Empirical experiment gives useful suggestions. In case of small credit data sample, SVM outperforms BP neural network in terms of prediction accuracy and generaliazation ability. In addition, the hybrid method of combining SVM and genetic algorithm can help bank to identify the critical factors affecting credit evaluation. These conclusions can be of great significance for domestic banks to evaluate personel credit.