Research On Support Vector Machine Classification Algorithm Based On Weighted Incremental Method

Support Vector Machine (SVM) is a new machine-learning technique which was put forward by Vapnik in the last century. It's the kernel content of statistical learning theory, has some advantages including simple structure and good generalization, and has been widely applied to many areas, such as pattern recognition, signal processing, automation, and data mining. However, the traditional support vector machine algorithm doesn't support incremental learning, which leads to its bad performance in large-scale data situation. So it has great significance in improving the classification performance of the incremental support vector machine algorithm both in theory and application fields.The main target of this thesis is to find a new SVM learning algorithm based on weighted incremental method above the current SVM algorithms, and this new algorithm will have a better classification performance in large-scale data situation. This new algorithm's innovation mainly includes two points: first, some sample data deviate from their categories because of noise and other uncertain factors; but the current incremental learning algorithms work on the assumption that all data are independent and has the same distribution, which is apparently unfair to the rest of normal data, therefore a weighted value for each sample data should be introduced. Second, during the incremental learning process, the thesis analyses the relation between the Karush-Kuhn-Tucker (KKT) conditions of SVM and the distribution of the training samples, and comes to the conclusion that besides the current support vectors, samples which either violate the KKT condition or fulfill the KKT condition but near the classification hyperplane are more likely to become the new Support Vectors after a new training procedure. With this algorithm, the useful training samples of importance are reserved and the mostly useless samples are discarded.The content included in the thesis can be summarized as follows:Chapter 1 introduced the SVM's research background and actuality, and then put forward the target of this thesis. Chapter 2 laid emphases on the theory based on SVM, described and compared three algorithms of SVM, which were the groundwork of next research works.Chapter 3 introduced the theory for Incremental SVM algorithm, analyzed and compared three classic incremental SVM algorithms.Chapter 4 analyzed the deficiency lied in the current incremental SVM algorithm, and put forward a new SVM classification algorithm based on weighted incremental method. The thesis then testified this new algorithm on the standard dataset.Chapter 5 summarized the main contribution in the thesis and discussed the next step for future relative research.