Study On Improved LMS-KNN Nearest Neighbor Classification Method

As one of the classical machine learning algorithms,the nearest neighbor classification algorithm is widely used in advertisements,chatting robots,network security,health care,marketing planning and so on in recent years.Because it is easy to implement,does not need to estimate the parameters and suitable for classification problems.A nearest neighbor classification method based on the local mean and class mean(LMS-KNN)is an improved algorithm for the problem that the K neighborhood classification is not sensitive to the outliers and does not use the sample global information.Although the LMS-KNN algorithm improves the classification accuracy and classification efficiency,the algorithm still has some drawbacks.The imbalance of the data will affect the classification accuracy of LMS-KNN.Simultaneously such as the value of the nearest neighbor value K,the selection of distance measure,the contribution of the sample point and so on.Therefore,in order to improve LMS-KNN algorithm classification accuracy,this paper carried out the following research work:1)We summarize several common neighbor classification methods,analyze their respective algorithms and introduce the classical optimization algorithm.2)In order to reduce the impact of unbalanced data on LMS-KNN classification accuracy.The preprocessing of data is carried out by iterative nearest neighbor sampling algorithm.The processed approximate equilibrium data set is classified by semi-supervised local mean and class mean algorithm.3)The parameters of LMS-KNN classification algorithm are determined by cross validation and traditional iterative algorithm.Firstly,the cross validation error of the classification algorithm is modeled,then this paper use objective decision-making information to determine the weight of class mean as a mathematical formula.The simultaneous method with step size optimization techniques obtains a more appropriate distance weighting.Under the rule of subjective and objective decision rules,the classification accuracy and classification efficiency of traditional algorithms are improved.4)In order to optimize how to determine the parameters of the LMS-KNN classification algorithm.Genetic algorithm can solve nonlinear and multi-objective and other complex optimization problems and do not rely on the specific areas of the problem.The paper proposes a GA based feature weighting algorithm named GA-LMSKNN.Using the class mean weight for initial population,classification error as evaluation function through the iteration of genetic algorithm to select the best class feature weighting.In comparison with other methods such as traditional KNN,LM-KNN and LMS-KNN,the experiments show that the GA-LMSKNN can get suitable feature weights,result in good classification accuracy In UCI dataset.