Support Vector Classification And Its Applications

Example-based classical learning is an attractive framework for extracting knowledge from empirical data,with the goal of generalizing well on new input patterns. The goal of classification is constructing an classifier though training points. The classifier can map training points to a class and can make prediction. Many real-world process may be solved by using example-based learning methods.When designing classifiers with learning methods,although more training samples can reduce the prediction error, the learning process can itself get computationally intractable. This issue is becoming more evident today,because there are many complex classification problems in real-world domains, such as medical diagnosis, needed to be solved.Support Vector Classification(SVC)[1] proposed by Vapnik is a new learning technique based on the Statistical Learning Theory. Due to the solid theory foundation and good generalization capability, it has drawn much attention on this topic in recent years. The quality and complexity of the SVC solution does not directly depend on the dimensionality of the input space. After nonlinearly mapping the input space into a higher dimensional space, called feature space,the SVC constructs an optimal separating hyperplane in this feature space. The explicit construction of this mapping is avoided by the application of Mercer's condition. Kernels that satisfy Mercer's condition and are commonly used in SVC are linear, polynomial,and radial basis function. The training of SVC is done by quadratic programming. This paper is an overview of the SVC. Given a training set of N data points {x_k,y_k},where x_k∈Rⁿ is the kth input pattern and y_k∈{1,-1} is the kth output pattern, if the points can be approximately parted by a line, the support vector method approach aims at constructing a classifier of the formthe classifier is constructed as follows. One assumes thatWhereξ_i is slack variables,ξ_i≥0.And the margin is (?).In order to maximize the margin, we minimum ||w||,Then we getwhereC≥0.Through solving the dual quadratic programming problem we introduce following theoremTheorem 1 Consider a linearly separable training sampleand suppose the parameterα^* solving the following quadratic optimization problem Thenwhere j satisfies 0<α_j^*< C.If the training points can not be separated by a line, then we can map the input space into a higher space,kernel spaceφ(x).In this space. (φ(x_i)·φ(x_j))=K(x_i,x_j),where K(x, z) is called kernel function.Theorem 2 Consider classifying a training sampleusing the feature space implicitly defined by the kernel K(x, z). and suppose the parametersα^* solving the following quadratic optimization problemThen f(x) =sgn(∑_i=1^l y_iα_i^*K(x_i,x)+b^*) where b^* =∑_i=1^l y_iα_i^*(x_i·x_j),and j satisfving 0<α_j^*eff.So we should chose function f = sgn((w·x)+b) that makes d_eff minimum. In fact, the SVC just totaly do this.When solving large problems,obviously we could solve convex quadratic programming to solve SVC, but in fact this kind of algorithm may be out of work because of storage and computation. So there are some more effective methods such as chucking, decomposing and sequential minimal optimization(SOM).Recent years, More and more people pay attention to SVC and work hard for it. So that some improving SVCs are given.Because standard SVC do not give direct meaning of parameter C. so it is difficult to choose value of it. Scholopf[29] proposed v-SVC in which v replace C, turn out that v has apparent meaning(?)it is the upper value of the rate between the number of error points and the number of total sample points. It also the lower value of the rate between the support vectors and the number of total sample points. So v can control the number of support vector and the error.J.A.K.Suykens and J.Vandewalle[7] formulate a least version of SVC for classification.They consider equality constraints for the classification problem with a formulation in least squares sense. As a result the solution follows directly from solving a set of linear equations, instead of quadratic programming. While in classical SVC many support values are zero,in least squares SVC the support values are proportional to the errors.Here we introduce a least squares version to the SVC classifier by formulating the classifier problem assubject to the equality constrains through Lagrange function the solution is givenIssam Dagher[8] proposed quadratic kernel-free non-linear support vector machine(QSVC). QSVC is proved to be put in a quadratic optimization setting. This setting does not require the use of a dual form or the use of Kernel trick.In the end,we give two examples of application of support vector classification for classification of Iris and the diagnosis breast cancer.