Study Of Support Vector Machines Algorithm Based On Statistical Learning Theory

Traditional statistics is based on assumption that samples are infinite, so are most of current machines learning methods. However, in many practical cases, samples are limited. Most of existing methods based on traditional statistical theory may not work well for the situation of limited samples. Statistical Learning Theory (SLT) is a new statistical theory framework established from finite samples. SLT provides a powerful theory fundament to solve machine learning problems with small samples. Support Vector Machine (SVM) is a novel powerful machine learning method developed in the framework of SLT. SVM solves practical problems such as small samples, nonlinearity, over learning, high dimension and local minima, which exit in most of learning methods, and has high generalization. Currently, being the optimal learning theory for small samples, SLT and SVM is attracting more and more researcher and becoming a new active area in the field of artificial intelligent and machine learning. This dissertation studies multi-output Support Vector Regression (SVR), multiclass SVM, support vector machines algorithm, and applications of SVM and SVR. The main results of the dissertation are as follows: 1.After the original formulation of the standard SVM is studied and analyzed, a new learning algorithm, Non-equidistant Margin Hyperplane SVM (NM-SVM), is proposed to handle some frequent special cases in pattern classification and recognition. The separating hyperplane of NM-SVM is not equidistant from the closest positive examples and the closest negative examples. 2. Support vector regression builds a model of a process that depends on a set of factors. It is traditionally used with only one output, and the multi-output case is then dealt with by modeling each output independently of the others, which means that advantage cannot be taken of the correlations that may exist between outputs. The dissertation extends SVR to multi-output systems by considering all output in one optimization formulation. This will make it possible to take advantage of the possible correlations between the outputs to improve the quality of the predictions provided by the model. 3.For the study of SVM training algorithm, training a Support Vector Machine requires the solution of a very large Quadratic Programming (QP) optimization problem. Traditional optimization methods cannot be directly applied due to memory restrictions. Up to now, several approaches exist for circumventing the above shortcomings and work well. The dissertation explores the possibility of using Particle Swarm Optimization (PSO) algorithm for SVM training. 4.For lager-scale samples, based on Rough Sets (RS) theory and SVM, an integrated method of classification named RS-SVM is presented. Using the knowledge reduction algorithm of RS theory, the method can eliminate redundant condition attributes and conflicting samples from the working sample sets, and evaluates significance of the reduced condition attributes. Eliminating the redundant condition attributes can cut down the sample space dimension of SVM, and SVM will generalize well. Deleting the conflicting samples can reduce the count of working samples, and shorten the training time of SVM. 5.The methods constructing and combining several binary SVMs with a binary tree can solve multiclass problems, and resolve the unclassifiable regions that exist in the conventional multiclass SVM. Since some existing methods based on binary tree didn't use any effective constructing algorithm of binary tree, several improved multiclass SVM methods based on binary tree are proposed by using class distance and class covering of clustering. 6.The study of SVM and SVR application. An approach based on voice recognition using support vector machine (SVM) is proposed for stored-product insect recognition. Adaline adaptive noise canceller is used as voice preprocessing unit, feature vectors are extracted from audio signals preprocessed of known insect samples, and used to train multiply SVMs for insect recognition. The operation is very convenient, only requiring the insect's audio signals collected by sensors without insect images or samples. Focusing on the difficulty of scattered data approximation, two methods of surface approximation based on SVR are presented, which have been applied to reconstruct temperature fields of large granaries.