Research On Algorithms For Feature Incremental Learning Based On Support Vector Machine

As the arrival of the information age, especially with the rapid development of network and the "information explosion", the traditional technology of information mining and knowledge accessing is facing great challenges. On one hand, we have a great deal of data, which contains information and knowledge of great potential value. On the other hand, the rate of information to update reaches an alarming degree. As a result, the data classification technology with incremental learning is becoming one of the key technologies in current information processing. Compared with the traditional technology of data classification, incremental learning has two significant advantages. Firstly, it can save memory space because there is no need to preserve the historical data. Secondly, it significantly reduces the follow-up training time because it can make full use of the historical results of the training.With the diversification of equipments and means for acquisition, one object can be characterized by more and more features. Using the structural parameters trained from previously training data together with the additional features to improve the classification accuracy is our main research subject.The ideas of our research are as follows. Firstly, the basic theory of support vector machines is introduced systematically. Then a feature incremental learning algorithm is proposed to deal with the case in which the dimension of new additional features is the same. An incremental max-margin algorithm is proposed to deal with new samples containing different dimensions of additional features. This thesis makes the following contributions.1. An optimal kernel function is proposed for a given training dataset. According to the theory of optimal kernel functions, we show that the optimal kernel fitting a given training set is the convex combination of basic kernel functions.2. An incremental feature learning algorithm based on Least Square Support Vector Machine is proposed to tackle with incremental learning problems with the same dimension of new features. Using historic structural parameters trained from the existing features, the algorithm only trains the new features with Least Square Support Vector Machine. Experiments show that our algorithm needs less training time and can achieve comparable or better performance than that of standard Least Square Support Vector Machine.3. A max-margin incremental learning algorithm is proposed to deal with incremental learning problems with different dimensions of new features. Using historic structural parameters trained from the existing features, the algorithm only trains different dimensions of new features with max-margin algorithm. Experiments show that our max-margin incremental learning algorithm demonstrates better performance than that of filling algorithms. What's more, the classification accuracy of our algorithm is more robust than that of other algorithms as the missing rate and the dimension of new additional features increase.Based on MatlabR2007a, all the algorithms above are simulated with the datasets from UCI and the results are analyzed in detail.