Bayesian Kernel Nearest Neighbor Classification Method

The design of the classifier is the core part of pattern recognition.There are many classification method of pattern recognition.Among them,nearest neighbor method is the most fundamental and intuitive classification method.Bayesian kernel nearest neighbor method(BKNN),which applied bayesian inference and kernel method to nearest neighbor method meanwhile,is an improved method based on nearest neighbor method.BKNN algorithm gave the results of probabilistic sense,completed the automatic selection of the parameters,and improved classification performance.In order to Make up the defects of Bayesian kernel nearest neighbor method and enhance the classification accuracy of the algorithm further,this paper is denoted to developing research of the new algorithm based on theory of Bayesian kernel nearest neighbor method.The main research work in this paper is as follows:(1)Research on editing Bayesian kernel nearest neighbor methodAlthough Bayesian kernel nearest neighbor method has so many advantages,this algorithm is at the cost of efficiency to improve the accuracy of classification.This paper deeply analyzes the editing algorithm put forward by Wilson to improve this situation,and combine the editing algorithm with Bayesian kernel nearest neighbor method,then put forward editing Bayesian kernel nearest neighbor method(E-BKNN)to accelerate BKNN method.This new algorithm firstly edit data set using editing algorithm,which could clear overlapped or noisy samples in the class boundaries to structure a high quality data set.Then,E-BKNN algorithm classify samples using BKNN with new data set.The theoretical analysis and experimental results show,compared to the original BKNN algorithm,E-BKNN has two advantages: the first one is that,E-BKNN has higher quality data set because of using editing method,which could lead to the improvement of classification accuracy;the other one is that,running time of E-BKNN has decreased because the number of data set decreased during the editing link,which improved the efficiency of the algorithm and increase the practicability of the algorithm?(2)Bayesian kernel nearest neighbor method based on optimal kernel distanceBayesian kernel nearest neighbor method still adopt Euclidean distance as metric in kernel space.However,in high dimension space,the hypothesis that class posterior probability is constant is false.Besides,for some practical situation,the number of useful sample is small,so the classification accuracy of Euclidean distance is barely satisfactory.This paper generalized the optimal distance put forward by Short to the optimal kernel distance,and applied the optimal kernel distance to BKNN algorithm,then put forward Bayesian kernel nearest neighbor method based on optimal kernel distance(opt-BKNN)algorithm.This algorithm firstly construct a small neighborhood using Euclidean distance,then calculate the optimal kernel distance in this neighborhood,finally classify samples based on the optimal kernel distance.Through theoretical analysis and simulation experiment,we can drop the conclusion that the new algorithm improved classification accuracy availably.Besides,opt-BKNN only introduce one external parameter,neighbor parameter,m.In the last,this paper discuss the law of the parameter via simulate experience.