Application And Research In Support Vector Machine

With the rapid development of IT and the widespread popularity of the Internet, the amountof generated data is increasing daily. However, how use these mass data to discover usefulinformation to help people to make the right decision and forecast the unknown phenomenon hasbecome a very urgent subject. As a result, machine learning techniques came into being in suchdemand.Statistics is the theoretical basis of the existing machine learning methods. Even though thetheoretical system of traditional statistics is completed, it needs to know the form of thedistribution of the sample in advance, and requires samples to be infinity. However, the numberof samples is limited in practical applications, and the results are unsatisfactory in someapplications. Based on this, the statistical learning theory is a specialized discipline whichexplores the law of machine learning in the case of finite sample. From the 1960s, Vapnik andothers started to focus on this research. With the theory’s continuous development, a relativelycomplete theoretical system is gradually formed.Vapnik proposed support vector machine (Support Vector Machine, SVM) based onstatistical learning theory. SVM overcomes the defect that traditional machine learning methodsonly consider the empirical risk minimization (Empirical Risk Minimization, ERM). Moreover,SVM adopts structural risk minimization (Structural Risk Minimization, SRM) principle to learnthe model. By solving the convex quadratic programming problem (Quadratic of Programming,QP), the global optimal solution can be obtained. Theoretically, SVM overcome local optimalvalue，over fitting and other issues that the neural network faced. It not only could solve thelearning problems given little samples but also maps the data to high dimensional feature spaceby introducing the kernel function, which overcomes the "dimension disaster" brought about bythe high-dimensional data. These advantages of the SVM have become new hotspots in machinelearning field.There are still many problems exist in SVM, however, they are also the researchfocus. The main problems faced as follows:(1) Parameter selectionNo matter for SVM classification or regression, the improperly selected parameters willhurt the generalization performance. Therefore, choosing proper parameter is particularlysignificant, such as penalty parameter C, which will produce over fitting if C is too big, and viceversa. Currently, Parameter choosing has attracted widespread attention, and there areevolutionary learning algorithm, the grid algorithm, and cross-validation methods. (2) Complexity problemSince SVM has been proposed, the improvement in complexity has been the research focus.SVM can be attributed to a quadratic programming problem. However, the number of variablesis equal to the number of samples, which leads to time-consuming owing to the traditionalquadratic programming when samples are massive.(3) Kernel computation complexityTo calculate the kernel function is not easy in terms of Large-scale data, which requires thatall samples operate the inner product. For example, in the SVM nonlinear transformation, thekernel matrix calculation will be time-consuming if the sample size is too large.Main innovations are proposed in this paper as follows:(1) A new algorithm (AGA-SVR) is proposed for effectively solving the parameter selection insupport vector regression. The chromosomes’diversity is improved by timely increasing thechromosomal aberrations’probability in the algorithm, which overcomes the defect thatindividual tends to precocious easily in standard genetic algorithm. As a consequence, this couldincrease the chance of obtaining the global optimum. AGA-SVR is applied to the Shanghai indexforecasting, which testify that it is superior to the standard genetic algorithm and the classicalgradient descent algorithm.(2) A new algorithm TDMSVM (Twin Distance of Minimum and Maximum Support VectorMachine) is proposed. Two optimal hyper planes are obtained by solving eigen-equation, andhyper planes are characterized by minimizing the average distance to one class whilemaximizing the average distance to another class. Theoretical analysis and experimental resultsshow that TDMSVM has the following advantages: low time complexity, without requiringregular, which improve the generalization performance, and the matrix’s singularity is overcome.(3) A new fast classification algorithm HSSVM (the Hyper the Sphere Support Vector Machine)is proposed. HSSVM uses two hyper spheres to respectively fit two class samples, andclassification model just requires the parameters which are sample mean and variance.Themodel’s training time complexity reduced to linear O(m) under the condition that no hurting incorrect classification rate. More importantly, the model robust to unbalanced data sets.Theoretical analysis and experiments show that the algorithm is a fast and efficient classificationalgorithm.(4) A new Nystr m algorithm is proposed by combining the random Nystr m and PrincipalComponent Analysis. The method could speed up the kernel methods’computing, and theoreticalanalysis and experimental results show the algorithm reduces the computing time while ensuringapproximate accuracy.