A Sparse Proximal Support Vector Machine In Classification Problems

Classification problem is the central problem in machine learning. It’s widely used in various fields, such as spam filtering, image classification, pattern recognition and cred-it rank evaluation etc. The purpose of classification is to assemble objects into different categories or assign new objects to the classes which we already knew. With the advent of the era of big data, plenty of specific classification and prediction methods have been mentioned. Support vector machine (SVM) is the most widely used method due to its ex-cellent small sample learning ability as well as advantages in solving the nonlinear, high dimension, local minimum problem.However, with the development of technology, and the higher dimensions of the data we obtained, how to get the sparse solutions and reduce the data dimensions turns to be the key research topic. Motivated by the fast computational efforts of proximal support vector machine (PSVM) and the properties of sparse solution yielded by l1-norm, in this thesis, we adjust cardinality constraints into the objective function through penalty parameter, and furthermore relaxed it as l1-norm, thus leads to the l1-l2 regularized sparse PSVM. Com-pared with the original PSVM, the proposed model can acquire solution that is sparser while guarantee the accuracy of classification, extract valid information from massive data, reduce the data dimensions and computing time.To solve this model, we convert the l1-l2 regularized sparse PSVM into an equivalent form of l1 regularized least squares (LS), which can be solved by a specialized interior-point method proposed by Kim et al that uses the preconditioned conjugate gradients algorithm to compute the search direction. The advantage of this specialized interior point method is that it can be used to solve large-scale problems.To testify the effectiveness of the model, we implemented the l1-l2 regularized sparse PSVM by a group of two-dimensional binary-classification and multi-classification datasets respectively, which later shows excellent results. Then we take more real datesets from the University of California, Irvine Machine Learning Repository (UCI Repository) for further numerical tests. The numerical results shows that, for binary linear classification problems, the l1-l2 regularized sparse PSVM achieves not only better accuracy rate of classification than those of GEPSVM, PSVM, and SVM-Light, but also a sparser classifier compared with the l1-PSVM. For multi-class classification problems, l1-l2 regularized sparse PSVM also performs well than some muli-category PSVMs.