Classification Algorithm Research Based On Twin Support Vector Machine

Classification technology is the development of important areas of machine learning and the basis of artificial intelligence. Classification divide the data sample into one or a few predefined classes, is a supervised learning method. Classification is not only an important element of pattern recognition, but also awareness of all things base. Support vector machine (SVM) [1] [2] [3] is a ma-chine learning method built on the basis of statistical learning theory, which uses the structural risk minimization criterion. The main advantages are:learning problems will be reduced to a convex quadratic programming problem and it will get the global optimal solution in theory. SVM performs better upon machine learning on small samples, non-linear classification, over fitting problem and "the curse of dimensionality" problem, it has now become a mainstream on the clas-sification algorithm. But it is not a universal algorithm, it plays not good on non-equilibrium problem in general, difficult to deal with cross-data the initial distribution of the data or the data mapped after.Data cross-distribution is similar to the type of the letter X, which can not be separated linearly. While SVM with non-linear classification approach is usually the introduction of relaxation factor for each sample, at this time algorithm certain some samples of the original set singed wrong label and correct it. This approach can deal well with the non-linear classification problem with little signed wrong samples, but not well with cross-data set because of a number of signed wrong samples, and actually this treatment is contrary to the original distribution of the cross-data set, the classification results will be not good.In 2007, Jayadeva et.al proposed Twin Support Vector Machines (TWSVM) [24] for solving binary classification problem. It construct two hyper-planes for each class to make it as far as possible away from the fitting of such samples in that class while others, these two hyper-planes are the classification planes, a new sample will be predicted in the class which hyper-plane is closer to it. According to this idea, TWSVM can deal very well with the cross-data set.Non-equilibrium problem mainly refers to the samples in one class or some more less than others. This problem has been recognized as a difficult problem in machine learning. Their difficulties are mainly the characteristics of the sample set itself, the uneven distribution of samples makes it not well reflect the actual distribution of the entire data set. SVM in dealing with such issues often classifies wrong on the class which has fewer samples.We applies TWSVM to non-equilibrium problem. Experimental evaluation used the ROC curve method instead the comparative accuracy between SVM and TWSVM. The ROC curve method is not sensitive to the distribution of categories and can better respond to the classification ability on non-equilibrium problem. The area under the ROC curve (AUC) represents the classification capability of the algorithm. In our experiments, the value of AUC for each data set of TWSVM generally is greater than SVM's, this show the advantage of TWSVM on the non-equilibrium problem.Multi-classification problem defines as the classification problem with more than two categories. Multi-classification problem is usually translated into many binary classification problems to solve, this transformation will be more non-equilibrium binary classification problems. TWSVM with the excellent perfor-mance in dealing with the issue of such problem should have a good effect. But we do not directly transform the multi-classification problem into binary classi- fication problems, instead, we proposes a multi-classification algorithm based on TWSVM. our algorithm of multi-classification problem for each type of samples constructs a hyper-plane, making such samples in this class as close as possible to the hyper-plane, while the other class samples away from the hyper-plane. This get the same number and type of the convex quadratic optimization problem, each optimization problems are less than a support vector machine solution of quadratic optimization problem required by the scale. And then, we introduced the kernel method into our algorithm and extended it to non-linear algorithm for the multi-classification.Linear TWSVM multi-class classification algorithms as follows:Algorithml1.Given training set X and categories Y, select the proper penalty param-eters Ci,i=1,2,…, N.2.For category k=1, keep it as set A, the others as B, solve the problem and get a. In terms of this, construct the approximal hyperplane f1(x)=w1x+b1 of the first category.3.Increase k from 1 to N; get the approximal hyperplanes fk of every cate-gory, Last, construct the decision function4.Put the test sample into the decision function (2), and then the forecast category is identified.We can obtain all of other approximal kernel surfaces of other categories through the same process. The test sample is predicted in a manner similar to the linear case: Algorithm21.Given training set X and categories Y, select the proper penalty parame-ters Ci,i=1,2,…,N and the kernel K(x1, x2).2.For category k=1, keep it as set A, the others as B, solve the problem and getβ. In accordance with the corresponding formula, construct the approx-imal kernel surface F1(x)= K(xT, CT)w1+b1 of the first category.3. Increase k from k=1:N, get the approximal kernel surfaces Fk of every categories. Last, construct the decision function4. Put the test sample into the decision function (4), and then the forecast category is identified.Finally, we take the new multi-classification algorithm through some exper-iments with the UCI data sets [41] to prove its outreach capacity both of the overall accuracy and the accuracies of each class. According to the experiments, out algorithm gets higher accuracies than SVM multi-classification algorithms almost four percent.