Support Vector Machine Algorithms With Exponential Gradient Updating

The Support Vector Machine (SVM), proposed by Vapnik and others, is a newclass of machine learning method based on statistical learning theory and structural riskminimization principle. Setting optimization, kernel function, the best generalizationability and other characteristics in one, it has a very good learning performance andgeneralization capabilities, and has become popular research field of machine learningwith many successfully application. Support vector machine can be attributed to solvingthe quadratic programming problem. In large-scale practical problem, due to intensivematrix computation, the speed of the quadratic programming remains slow, whichaffected the application of support vector machines at a large extent. Thus, on thepremise of little change of classification accuracy, it has great significance inaccelerating the SVM training both in theory and application fields.This article introduces the exponential gradient descent update, which is amultiplicative update. Theoretical analysis and experimental results prove that theconvergence rate of exponential gradient descent update is faster than that of traditionalplus gradient descent update when the learning machine is strongly sparse. Based on theperformance, we introduce an algorithm based on exponential gradient update rule tosolve the hard margin SVM. In this algorithm, all the variables are updated in paralleland the objective function of quadratic programming is down to its global optimumquickly. Furthermore, we put forward an algorithm based on exponential gradientupdate rule to solve the soft margin SVM. Before updating, we linearly transform themixed constraint quadratic programming and remove the box constraints. Fromtheoretical analysis and simulation experiments, this algorithm can remarkably reducethe computation complexity and accelerate SVM training.