Research On The Incremental Learning Algorithm For Support Vector Machine

Support Vector Machine (SVM) proposed by Vapnikin the1990s is a new tool tosolve machine learning problems with the help of optimization theory. It has manyadvantages in solving the small samples, non-linear, and high-dimension learningproblems. It can handle regression and pattern recognition successfully and has beenwidely used in handwritten digit recognition, face recognition, text classification, andreview of prediction andso on.The learningdata are always increasing step by step in some practical problems. Theincremental learning algorithm can make full use of historical information, reduce thetrainingscale greatlyandsave the trainingtime, so it is an effectivemethod tosolve theproblems with accumulated training samples. By analyzing the existing incrementallearning algorithms for SVM, we propose a new incremental learning algorithm basedon the center distance ratio and the center density. At the prepossessing stage, thecenter-distance ratio of every sample is defined as the ratio of its self-center distance toits mutual-center distance, then some samples with the higher center-distance ratio areselected as the trainingset to replacethe whole trainingdata andtrain. Whilesome newincremental samples are coming, the new training set includes the previous supportvectors, the new incremental simples violating the KKT condition of the previouspattern and a part of well-chosen non-support vectors. To well-choose the non-supportvectors into training set, we first find the number of some special samples whosedistance to their center are less than a certainthreshold, and definethe center density asthe ratioof the number of these samplesto the whole,then we choose some non-supportvectors into the training set according to their center densities. Experiments show thatthe new algorithm not only reduces the computation time, but also improves theclassification precision.