Theory And Algorithms For Nearest Neighbor Method And Multiple-view Learning

This paper reports the research on theory and algorithm for nearest neighbor method, multiple-view learning and ensemble learning. Improvements on all of them have been carried out.In real applications, nearest neighbor method is a very useful and widely-used ap-proach. However, it still has its shortcomings because it does not utilize all the information of the neighbors in the process of prediction. In this paper, traditional nearest neighbor method is improved to be a new method in which not only label information but also dis-tribution information of the nearest neighbors is utilized. Moreover, a centroid point is constructed according the nearest neighbors, and the distance between the query point and the centroid instead of the distance between the query point and the real neighbors is used to predict the hypothesis. Thus, both label and distribution information is utilized to carry out the prediction. In experiments, both Euclidean distance and Mahalanobis distance are employed to verify the proposed method on 12 real-world datasets. The empirical results suggest that the centroid nearest neighbor method can significantly outperform the traditional methods.In recent years, multiple-view learning has drawn much attention in the field of ma-chine learning. Multiple-view learning just utilizes multiple views to infer the inner pat-terns of the dataset under certain training strategy. Theory and practice have proved that multiple-view learning can significantly improve the learning performance. Moreover, some research has shown that combining multiple learners can also significantly improve the learning performance. However, there is no related research involving in combining multiple-view and multiple-learner. Part of this paper focuses on this point. And a method named multiple-view multiple-learner approach is proposed. It is also extended to solve semi-supervised learning and active learning problems. Moreover, a measurement which can calculate the disagreement on the prediction between views is proposed. From the empirical results, the proposed methods are very useful.There is a bottleneck in the field of multiple-view learning. Although multiple-view learning can significantly improve the performance of learning, lots of problems do not have multiple views. In other words, multiple-view learning method can not be directly used into single-view problems, which greatly limits the application domain of multiple-view learning methods. In order to overcome this problem, a view-generating method is proposed in this paper. Principle component analysis method is employed to create addi-tional views for the single view problems. Moreover, different view-generating strategy is proposed corresponding to the problems with and without high dimensionality problems. The proposed method is lunched on many benchmarks, and the empirical results suggest that the created view is suitable for multiple-view learning.Ensemble learning is to learning a problem by combining multiple classifiers. In traditional ensemble learning, classifiers for ensemble are generated in parallel, which means more classifiers will cost much more time. Different from traditional method, a evolutionary ensemble method is proposed in this paper. In the proposed method, classi-fiers is generated in an evolutionary mode instead of in a parallel mode. Lots of real-world problems participate in the experiments, and it can conclude that when the necessary as-sumptions hold, this kind of ensemble can significantly improve the learning accuracy.