Researches On Nonparallel Hyperplanes Support Vector Machines For Pattern Classification

Classification problem is one of the basic problems in artificial intelligence, pattern recognition,and machine learning. Support vector machine (SVM) proposed in1995is a powerful classificationtool for its minimization of the structural risk, avoiding the curse of dimensionality, and skillfullyusing the kernel technique. This paper reviews the proposed SVMs for pattern classification, andsummarizes them as diferent types by the hyperplanes in SVMs if they would be parallel and if theyfit the samples well as follows:(i) parallel separating hyperplanes SVM, e.g., C-SVM and ν-SVM;(ii) parallel fitting hyperplanes SVM, e.g., least square SVM and proximal SVM;(iii) nonparallelseparating hyperplanes SVM (NSSVM), e.g., parametric-margin ν-SVM and parametric-margin twinSVM;(iv) nonparallel fitting hyperplanes SVM (NFSVM), e.g., generalized eigenvalue proximal SVMand twin SVM. Parallel hyperplanes SVMs construct the classifier by a pair of parallel hyperplanes,which separate the training samples or fit them. However, in practical, parallel hyperplanes SVMsare not always work well on the datasets in the world, e.g., cross plane dataset, heteroscedastic noisedataset, and large dataset.Due to the nonparallel hyperplanes SVM can deal with some datasets which parallel hyperplanes SVMs can not, we pay more attention on the nonparallel hyperplanes SVM and proposed four new nonparallel hyperplanes support vector classifiers in this paper as follow:(1) smooth twin parametric margin support vector machine, which model is constructed by solving following: see the table of notations for more details.It applies smoothing technique into twin parametric margin support vector machine, owns a very fast learning speed by solving two quadratic programming problems with non-constraint;(2) proximal parametric margin support vector machine, which model is constructed by solving following:It improves the definition of the parametric margin in the parametric margin support vector machine so that it needs to solve a system of linear equations instead of a quadratic programming problem, has the unique model solution and works well on the dataset with heteroscedastic noise;(3) least square twin parametric margin support vector machine, which model is constructed by solving following: It introduces the idea of least square into twin parametric margin support vector machine and modifies the metric of the penalty of the sample, fits the samples well on the dataset with different distributions;(4) regularized projection twin support vector machine, which model is constructed by solving following:It introduces the regularization terms into the projection twin support vector machine such that it implements the minimization of structural risk, has the unique model solution.Furthermore, we design the probabilistic output for twin support vector machine and propose the model selection for twin parametric margin support vector machine in this paper. These model frames fill in gaps in the probabilistic output, parameter selection, and feature selection for nonparallel hyperplanes support vector machines. Generally speaking,(i) the classification methods mentioned above are hard classifiers, which output binary class labels, so we design a probabilistic output model for nonparallel fitting hyperplanes SVM by introduc-ing the definition of the cross separating hyperplanes, which help us define the degree of membership function and thus obtain the probabilistic function by minimizing the likelihood function;(ii) for model selection, a heuristic algorithm is introduced, and we design a frame to do the feature and parameter selection simultaneously, which is more faster and effective compared with traditional grid parameter selection. By the way, the above two model frame can be transformed to other NSSVM and NFSVM modelswithout any difculties.