An Approach To Incremental Support Vector Machine Learning Based On Artifical Neural Network

In recent years, support vector machine (SVM) has been successfully used as a classificationtool in a variety of areas in machine learning because of its solid theoretical foundations that it hasimplemented structured risk optimal. The original problem of SVM is a quadratic programming(QP) problem, and the result of which is reduces to a linear combination of training examples. Inthe era of Internet, the problems we encounter usually come with large scale data. However, SVMsuffers from the problem of large memory requirement and CPU time when trained in batch mode.The development of incremental SVM learning algorithms will be instrumental in practicalproblems.In the thesis we proposed a novel approach for incremental SVM learning based on BPneural network. In our algorithm, the hidden layer of the network is determined by the result SVM,namely the number of units in the hidden layer is the equal to the number of support vectors, andthe connection between the input layer and hidden layer is determined by the kernel functionchosen. When resolving the weights of connections between the hidden layer and the output layer,we used the stochastic gradient descent algorithm as training the perceptron. We also proposedtwo improvements: using the hinge loss function instead of square error loss function, and pickinginstances using methods similar to Condensed Nearest Neighbor. Theoretical analysis andexperiments on real world datasets both indicates that our algorithm performs similar to SVM,while requiring much less memory and time usage.