Researches On Incremental Support Vector Machine

Support vector machine (SVM), based on statistical learning theory, is a universal and efficient machine learning method. Because of its solid theoretical foundation and good generalization performance, it has been widely applied to many fields. But it solves a convex quadratic programming problem essentially, and then needs large-scale matrix operations. Therefore, it is only applied to small-scale data processing problems. For large-scale data, the matrix operation will be very complicated and then the learning efficiency becomes lower. The incremental learning method joins one sample or a batch of samples to learn in each cycle, and then the problem can be reduced from large-scale to a series of sub issues. This thesis proposes two incremental support vector machine (ISVM) models according to whether the samples have the label, based on the incremental learning technology. The proposed two models can improve the performance of SVM processing the large-scale data effectively. The main research works include:(1) Researches on the incremental learning process for the labeled sample. In this process, if the selection of the incremental sample is incorrectly, it may reduce the learning ability and generalization performance of the model. But the general algorithms of selecting the incremental sample are often random, or too complicated. To solve this problem, this thesis proposes the PISVM model based on the probability density distribution. The proposed model choices the incremental samples containing much important classified information (they might be support vectors). When the sample’s predicted value and actual value are inconsistent, it will be added into the training set, which accelerates the convergence speed of the model. The experimental results on UCI benchmark datasets demonstrate that the proposed PISVM model can maintain the generalization capability and further improve the learning efficiency.(2) Researches on the combinatorial semi-supervised support vector machine (S3VM) model including some labeled and much more unlabeled samples. The S3VM model has to select the optimal label as its final label from the combination labels of all unlabeled samples. The key problem is high computation complexity. To solve the problem, this thesis introduces the incremental learning method and proposes IS3VM model. This method reduces the computation complexity by labeling in batch of the large number of combinations of unlabeled samples. The unlabeled samples in the classification margin are selected to label and added into the training set to correct the model. At same time, the label which makes the margin largest is selected as its final label. In so doing, the correctness of the label can be ensured, and thus the accuracy of the model.This thesis presents two models PISVM and IS3VM. The method of selecting incremental samples of the ISVM model itself and the learning efficiency can be improved, and the application of ISVM in the field of semi-supervised learning is expanded as well. The obtained results of the thesis will be significant for application researches of SVM.