Research And Development Of Twin Support Vector Machine

Machine learning is a cross field of cross discipline, which is rise at the beginningof this century relate to statistics, matrix theory, optimization theory etc. The basicproblem of machine learning including the pattern classification problem, regressionand clustering problem, et al. The support vector machine (SVM) theory which pro-posed by Vapnik et al. in1995because of its theoretical foundation of structure riskminimization, each field characteristics of avoiding the curse of dimensionality problemand advantage in small sample has been widely reached and developed.During the year of2005to2007, the generalized eigenvalue proximal support vec-tor machine (GEPSVM) and twin support vector machine (TWSVM) were publishedon the top journals of artificial intelligence, which represents the parallel hyperplanethought of SVM transfer to the more complex nonparallel hyperplane. For the twononparallel hyperplanes SVM thought is proposed in classification problems, this dis-sertation firstly reviews these algorithms in pattern classification problem. Then, wehave researched the idea of TWSVM carefully and extend it to the basic problems ofmachine learning, which fills in gaps in these basic problems.GEPSVM completely view of the whole sample equally, which have the same efortto module structure for the sample in the diferent position of vector space. HoweverTWSVM have the same efort on similar sample, but have the less efort on the nosimilar sample, in that respect TWSVM and GEPSVM are diferent. Because of theoutstanding performance of TSVM of classification ability, this dissertation focuses onthe research of the TSMV concept, then extended this concept to other basic problemsin machine learning which make the advantage of the thought of SVM deserved betteruse. In regression problem, there are two types of twin support vector regression, oneis “twin” form and the other is “ε twin” form. We consider the aspect of the anti-noise of ε twin support vector regression, and propose two novel regression algorithmsby using L1norm and weighted slack vectors respectively. The former changes thepenalty of the whole dataset by L2norm instead of L1norm, therefore, it reducesthe influence of the noise totally, and the original quadratic programming problembecomes to a piecewise linear programming problem. By some algebraic techniques,the piecewise linear programming problem can be solved as a linear programmingproblem by simplex method. The latter considers the diference between the samplesand the noise in the vector space, so that it can reduced the influence of the noise byadding diferent slack weights on diferent samples. Both improved methods in dealingwith some special samples were all played a good role in the anti-noise function, andeven can be used together which is our next research direction. The experimentalresults show the efectiveness of these two methods in anti-noise.In clustering problem, there are rarely researches about GEPSVM and TWSVM.Therefore, we introduce the above ideas into clustering and propose two new clusteringalgorithms. The two new algorithms to fills in gaps in TSVM and GEPSVM in theresearch of clustering. In addition, especially for k-mean and k-plane clustering algo-rithm random initialization sample cluster categories of uncertainty, we give a samplecluster class initialization method based on p neighbor graph. Experimental resultsshow that the initialization method can not only make the presented clustering algo-rithm has a very high stability, but also can improve the clustering efect of clusteringalgorithm in a certain extent.In feature selection problem, owing to the single weight feature of SVM, it is d-ifcult to do the feature selection on the double weights feature of TWSVM. In thisdissertation, in spirit of the double hyperplanes in TWSVM, we propose a novel fea-ture selection algorithm by introducing the feature selection matrix into the TWSVMmodel which changes the feature selection problem into matrix optimization prob-lem. The feature selection based on TWSVM (include linear classifier and nonlinearone) is solved skillfully. These algorithms proposed in this dissertation are tested inexperiments and the results confirm their efectiveness.