Comparative Study Of Several Feature Weighted Support Vector Machine Methods

Support vector machine (SVM) is a new machine learning method based on statistical learning theory, which seeks the best compromise between the model complexity and learning ability according to the limited sample information available so that a better generalization ability can be obtained.The traditional SVM considers all the features of equal importance, however, if the sample set contains certain features which have little relevance or even irrelevant with the target function, they will affect the generalization ability of classifier to some extent. In some real databases there are indeed cases where some features of the classification make great contributions, while others have little contributions to the classification, therefore, the feature weighted SVM algorithm appears. Feature weighting is to choose a number in interval [0,1] to show the importance of the features, and the more important the feature given the greater weight.In this thesis, a number of feature selection methods are applied to the classical SVM, and five feature weighted SVM algorithms are proposed. The feature weighted methods are as follows Gain ratio, Symmetric uncertainty,x2test, Gini index as well as Relief-F algorithm. Then a definition of relative margin is proposed in this thesis, and proves under certain conditions the bigger the relative margin the better the SVM generalization ability. Finally, some experiments are done on one synthetic data set and eight real data sets, the experimental results are analyzed and compared. Compared to the classical SVM algorithm, these five feature weighted SVM algorithms proposed in this thesis improve the classification accuracy of classifier to some extent, and decrease the number of supportvector s in most data sets.