Semi-supervised Twin Support Vector Machine

Twin Support Vector Machines(TWSVM) is proposed in 2007, which is similar to Support Vector Machine(SVM). However, the training time of TWSVM is reduced to 1/4 of SVM. TWSVM’s idea is based on Proximal Support Vector Machines(PSVM) and Proximal Support Vector Machines based on the generalized eigenvalues(GEPSVM). Similar to SVM, TWSVM also has a solid theoretical foundation and strong generalization. Because of its superior performance, TWSVM has become a hot research topic in the field of machine learning. Scholars have also proposed a variety of optimization and improvement on it. However, the standard form of TWSVM only applies to supervised learning. Large amount of data generated in real life is unlabeled, and the standard form of TWSVM cannot make good use of these data to improve its learning ability. When there are a small amount of labeled data and a large amount of unlabeled data, semi-supervised learning method can help TWSVM solve these problems and improve its performance. Combining the semi-supervised learning and TWSVM to change or improve the learning behavior will be an interesting work, and the main contents are as follows:Firstly, this paper adds these unlabeled data on the optimization function of the standard form of TWSVM, and proposes the original model of semi-supervised TWSVM. The entire expression of standard TWSVM must use labeled data to compute. After doing some derivation on the formulations of standard TWSVM, the unlabeled data can be reflected in the formulations, which means that it can use these unlabeled data in the process of solving. This can also be considered to be the original model of semi-supervised TWSVM.Secondly, considering that it’s complex to solve the original model, this paper uses Manifold Regularization framework to propose a Laplacian Global Preserving Twin Support Vector Machine(LapGTSVM). Manifold Regularization framework is a hot topic and can be effectively utilized to semi-supervised learning. Laplacian Twin Support Vector Machines(LapTSVM) is the first machine learning method which applies TWSVM to semi-supervised learning by using Manifold Regularization framework. The LapGTSVM in this paper embed the global structural information of data in LapTSVM, so that the final classifier can fully consider the global and local information of data. Compared to LapTSVM, LapGTSVM has a great improvement in stability and performance.Finally, by using Bagged Cluster kernel, TWSVM can make full use of the unlabeled data, and this paper proposes TWSVM Based on Bagged cluster kernel for semi-supervised(Bagged-TWSVM). In this method, the main idea is to cluster kernel of Semi-supervised Kernel. Cluster kernel can readjust the information of the similarity between samples, which makes the degree of similarity increasing when the samples have been clustered in the same class. Bagged-TWSVM can make full use of the information of unlabeled data, thereby increasing the classification accuracy.