The Research Of Support Vector Machine Incremental Learning Algorithms

Support Vector Machine (SVM) is proposed by Vapnik in 1995 for the classification and regression in statistical learning theory. Because of its global optimum and good generalization, it has become a popular ongoing research focus of machine learning and pattern recognition. In recent years, both the theory and application have been extensively studied, and achieved some important achievements. However, the time and space complexity of support vector machine is a bottleneck in processing large-scale data. Thus, this thesis will give some researches about incremental learning based on SVM to overcome the shortcomings of SVM in accuracy and speed of classification.This research works mainly around the following aspects:First, outline the basis content of our research: statistical learning theory and support vector machine approach, which are the groundwork of the follow-up study works.Several incremental learning algorithms based on SVM are described and compared, characteristics and disadvantages of several representative algorithms based on support vectors and KKT conditions are analyzed. Then geometric knowledge from Hyper-sphere SVM and advantages in dealing with large-scale data from a gradual incremental SVM learning strategy, which provide a theoretical support for the following proposed improved algorithm.A gradual incremental learning algorithm based on hull vector SVM is proposed. In the incremental learning process, a set of hull vectors are extracted from original training samples to become new training simples firstly, then KKT conditions which is determined by initial classification to select a new sample set, effectively reduces the need time for original and new training samples. In short, the algorithm achieved selective forgotten and elimination in both historical training data and new training data, while still ensuring good classification accuracy. Simulations showed the effectiveness of the algorithm.The algorithm is applied in the handwritten digit recognition area.