Research On Support Vector Machine Models And Algorithms Based On Additional Information

Recently, exploiting additional information to improve traditional inductive learning has become a research topic of renewed interest in machine learning. In a data-rich world, there often exists additional information about training samples, which is not reflected directly in the training set. This additional information can be easily ignored by the standard inductive methods such as SVM. This additional information of training data brings new challenge to the traditional inductive learning methods such as SVM. There are two approaches for solving such problems, known as Learning Using Privileged Information (LUPI) and Learning With Structured Data (LWSD). An SVM-bascd optimization formulation under LUPI and LWSD setting is called SVM+. SVM+can effectively utilize this additional information to improve generalization. This thesis gives further study on SVM+, mainly includes the following four aspects.1. Currently, nearly all SVM+learning models focus on two or multiple classes classification problems. However, there is no research touching on the employment of the SVM+in one-class classification problems. The use of any type of additional information as part of the one-class SVM has not yet been performed. We aim to utilize the advantages provided by the SVM+and focus on the one-class classification problem. By embedding the additional information into the corresponding optimization problem, we derive a;v-SVM style SVM+framework for one-class classification. All modeling results suggest an improvement in generalization performance due to partitioning the data into several informative groups2. Although SVM and SVM+work well in practice, a closer look at such theoretical results reveals that the generalization ability of these methods is strongly linked to the margin as well as some measure of the spread of the data. Yet the algorithms themselves only seem to be maximizing the margin, completely ignoring the spread information. In other words, their solutions can easily be perturbed by an affine or scaling transformation of the input space. In the spirit of relative margin machine (RMM) and SVM+. we propose an improved model of SVM+. This new model takes into account not only the spread of data but also group information hidden in the data to improve generalization. All experimental results suggest an improvement of the new model in generalization performance.3. We propose a modified model of support vector machines plus(SVM+) inspired from the opti-mization of Fisher’s discriminant ratio, the so-called minimum class variance SVM plus (MCVSVM+). MCVSVM+has both the advantages of Fisher’s discriminant and SVM+. That is, MCVSVM+con-siders class distribution characteristics in its optimization problem and ensures separability. At the same time, It explores the additional information hidden in the data, in contrast to Fisher’s discrimi-nant that docs not ensure separability and to SVM+that takes into consideration only the samples that arc in the class boundaries. All experiment results have shown that it has an improvement in generalization performance.4. One of the main challenges in machine learning as in classical SVMs and relative margin machine (RMM) is that they require large training time for large-scale datasets as they have to optimize a computationally expensive cost function. In order to overcome this drawback, inspired by the twin support vector machine (TWSVM), we present a fast training algorithm for the relative margin support vector machine termed as relative margin twin support vector machine (RMTWSVM). Similar with TWSVM, RMTWSVM aims at generating two nonparallcl hypcrplancs such that each one determines the positive or negative hyperplanc of the separating hypcrplanc, and it solves two smaller sized QPPs instead of solving a large one as in the RMM. So, the RMTWSVM can obviously reduce the computational cost of RMM.