Study On The Method Of Non-linear Classification Base On The Contraction Of The Closed Convex Hull

Support vector machine is a brand new method of machine learning developed from statistical learning theory, it solved the problem of statistical learning in condition of small sample data, so it have the better generalization ability and extension property. By constructing the kernel function, support vector machine is a learning machine which is using the linear method solving the non-linear problem. Kernel method for pattern analysis derived from the kernel function as the core section of support vector machine, which have being the powerful foundations of mathematics and research platform of studying the non-linear learning problem. Kernel theory reduced the complexity of studying on the non-linear learning problem and directly converted the non-linear learning problem into the linear learning problem in feature space, so kernel method for pattern analysis became the methodology guiding researcher studying on non-linear learning problem.Via in-depth researching on support vector machine and kernel theory, and comparing the multi-classification of support vector machine, the main achievements in this paper are described as the following.Firstly, the bisecting-nearest-point method is extended and transformed to a non-linear classifier method utilizing the kernel theory. Both of the bisecting-nearest-point method and support vector machine have the same classification ability and extension property, they have the same hyperplane for the same sample data of two class, but the algorithm of bisecting-nearest-point possesses more intuitionistic geometric meaning and lower calculation complexity.Secondly, for the non-linear inseparable problem, the contraction of the closed convex hull is extended and transformed too. The contraction of the closed convex hull is a mathematical tool of data preprocessing which shrink two classes data finitely to their mean centre, initially it used to transform two classes inseparable data into separable data to the linear problem, then the non-linear kernel form is proposed in this paper. Compare with support vector machine which solve inseparable problem using the soft interval theory, the bisecting-nearest-point method based on the contraction of the closed convex hull have higher superiority and practicality, because the method of contraction of the closed convex hull can reduce error rate to zero in training using appropriate shrinkage ratio.Thirdly, the algorithm of judging the overlap of the two classes of data is puts forwards and the separable metric matrix is constructed for the multi-classification, furthermore, this paper find out new conditions for the judgment of the constriction coefficient using the separable metric. For the different classification problem, what kind of classifier is chose need considering the distribution of the correspondent data, however the majority of the classifier have no this pre-information about the data distribution, so they have certain blindness. The separable metric proposed in this paper could judge if there is overlapped sample data or not between each class by constructing the separable metric matrix, at the same time the matrix give the depth of overlapping in each two classes. Via abstracting this pre-information, the blindness of choosing the classifier can be reduced and existing method of multi-classification can be advanced to improve their accuracy of classification.Fourthly, the multi-classification based on the decision tree method is advanced by the theory of contraction of the closed convex hull and the separable metric. Because the classification ability of the multi-classification based on the decision tree method is influenced by the structure of the tree, especially the choosing of the root node, however the common method is unrestricted for it, so the ability of classification of the tree could not improved apparently because of the blindness, via constructing the separable metric matrix in this paper, all the data of classes is contracted using the maximal depth of overlapping, two of the classes most easily separated are chose for up-node of the tree, constructing the decision tree in turns, so this method could reduce the error rate of classification and improve the classification ability of the tree.Fifthly, The research on the hand-written number and financial Chinese characters comparing with other multi-classification based on support vector machine proves that the algorithm's accuracy of classification and distinction.