Solving Non-Linear Classification Problems With A Sample Space Analyze Method

Support Vector Machine (SVM) is a general pattern classification method. Because of its powerful learning ability and good generalization performance, SVM has been applied to many different domain of pattern classification. The principle of finding optimized decision boundary give SVM excellent performance on linear separatable problems. However the performance cannot be guaranteed when solving some non-linear problems.To deal with large-scale pattern classification problems, a parallel SVM training approach that employ a part-versus-part task decomposition method, named Min-Max Modular SVM, has been proposed by B.L. Lu and his colleagues. With this method, a large scale binary classification problem can be decomposed into a series of smaller and independent sub-problems, which can be learned by SVM with parallel computers. And the resulting sub-problem classifiers can then be combined with Min and Max principles to produce an overall classifier to the original problem. By analyzing the distribution of training samples, we can not only divide the training samples, but also divide the sample space accordingly. Using this approach, a complex non-linear classificatioin problem in the global sample space could be divided into a series of simpler local classification problems, which can be learned better with SVM algorithm.Firstly, the thesis introduces SVM's optimal decision boundary theory, as well as the kernel method that enabling SVM to solve non-linear problems. Then discussed the reason why there could be further improvement when dealing complex non-linear problems.Secondly, the principle of Min-Max Modular methed is introduced. Differrent criteria of training set decomposition and their effect on classifier accuracy is discussed. The test phase of M~3 method usually costs more computational time than traditional SVM classifier. An optimization of test phase is introduced to speed up test phase.Then we paid more effort on discussing a sample space analyzing method that is proposed to improve the performance on non-linear problems. For a given binary classification problem, the method first divide the problem into a series of local classification problems in...