Research On SVM-based Local Weighted KNN Classification Algorithm

With the rapid development of information technology,data mining has a great potential in application.KNN is one of the most the important algorithms in data mining.The key idea of KNN is predicting the label of unclassified data by the train data around it.The label belongs to the majority label of neighbors.KNN algorithm is widely used in many fields because of simple,high robustness and good performance in multi-classification problems.In this paper,we make some research and analysis on the KNN algorithm and summarize some disadvantages:(1)In traditional KNN,the K is a global value,however,in uneven data set,the global K always performs worse.So it needs to suit the value K with distribute of the unclassified data.(2)Traditional KNN algorithm always chooses Euclidean distance as the metric to measure the similarity.However,the Euclidean distance metric ignores the differences between the attributes and shares the same weight to all attributes.In order to overcome these disadvantages,we propose a locally weighted SLKNN classification based on SVM in dealing with some drawbacks of KNN algorithm.In order to optimize to distance metric,we obtain an eigenvector by using SVM to classify the train data.Then the eigenvector is introduced in the distance metric between the unclassified data and train data to assign different weights to different attributes.Based on the optimized distance metric,we set different number K of neighbor to each unclassified data and set different weights to their neighbors.The value K and weights is calculated by SLKNN.Finally we obtain the estimate and predict its label.In experiments,we compare SLKNN with other algorithms,such as traditional KNN,ADPT algorithm,L-KNN and a local mean non-parametric classification.We choose several metrics,such as error rate and F-measure,to measure the performance and stability of the algorithm.The results show that SLKNN performs better.