Research Of The Method Of Adjustable Entropy Function Of SVM

SVM (Support Vector Machine), a new machine learning method based on statistical theory has been proposed in1992. In training procedure of the learning machine, the SRM (Structural Risk Minimization) criterion has been used to make it relatively simple mathematical form with intuitive geometric explain and good generalization ability. In the method, it transfer learning problem into a QP (Quadratic Programming) problem to ensure that the obtained solution is a global optimal one in the theory and to avoid local minimal problems arise. To ensure the learning machine’s good generalization ability, it maps source data into high-dimensional feature space by nonlinear mapping, and makes the data in high-dimensional feature space distinguish the unction category by linear method. Also, the problem of the dimension has been ingeniously solved to make the algorithms complexity have nothing to do with the sample dimension.As a new technology with several kinds of standards, many fields and skills of SVM have been researched or proposed such as the maximal margin hyperplane,Mercer core,convex quadratic programming,sparse solutions and slack variable, etc.. SVM technology has also been successfully applied in many challenge tasks such as predicting, classification and linear or nonlinear regression which are real world related problems.In the studies of SVM, many researchers proposed their new methods or algorithms, but all of these algorithms have some limits in the applied fields even though they maybe have some advantages at the same time. But all of them have the same feature that a QP or linear equation method is applied to deal with the most optimized questions. In pattern recognition (PR) field, QP dual technique is used to transfer optimized problem into simple constraining one under the high-dimensional feature space. Although, it is possible to process this kind of high-dimensional dual plan by the method of dissolving training or inputting orderly of the sample set. The generated algorithm not only owns the advantage of space saving, but also owns a higher improved efficiency of calculation. However, both the design and realization of the algorithm are much more complexity.The maximum entropy method prompted recently can solve the optimized questions effectively such as max-min or multi-constraint nonlinear mapping etc.. It is a new algorithm with rapid speed to converge, good stability of the number value and easy to be realized by computer languages. And the more, it is very effective and owns a high application value in solving large-scale and multi-constraint optimized questions and some non-differentiable optimized questions. Its disadvantage is that the accurate solution of the question can be obtained only when the parameter P becoming infinite, but, the overflow of the calculating value will be happened very easily when the P becomes larger.In the light of this disadvantage, in this thesis, the basis of SVM theory which includes statistical learning theory and optimization theory, the classification idea, method and regression principle based on SVM have been discussed and researched carefully. Based on the method of classification of SVC (Support Vector Classification) and the regression principle of SVR (Support Vector Regression), a new entropy function method with adjustable factor is proposed to solve the optimized problems of SVM. It can overcome the drawback of current entropy function method with adjustable factor that the P value must become to be infinite when obtain accurate solution. When it is applied to the problems of pattern recognition and regression of SVM, a better accurate classification result of SVC and a higher regression performance of SVR can be effectively realized under a low requirement of memory.