High-performed Kernel Classification Methods Based On Multi-kernel Learning

Classifier design is one of the main research directions in pattern recognition. An efficient design method of classifiers can make the recognition process to get twice the result with half the effort. Kernel methods are an efficient kind of classification methods. However, the con-ventional kernel methods are based on the single types of kernels, which often fail to describe the characteristics of patterns. Therefore, it is quite necessary to study the kernel classification methods based on multiple kernels. The learning algorithms for multi-kernel based classifiers are expected to describe the diverse characteristics of patterns as detailed as possible by means of the fusion of multiple kernel functions on purpose to process the multi-source and heteroge-neous data sets flexibly and stably.Because of a large amount of computational requirement in the fusing process of multiple kernel functions, the obtainable learning algorithms of multi-kernel classifiers often run up against such problems as high computational complexity and large memory occupancy. With the help of the Nystrom approximation and random projection methods, this thesis will study and design several types of efficient multi-kernel learning algorithms, and strive to reduce the spatial and time complexities and thus get high classification accuracy.The main work of this thesis is as follows.(1) A multi-kernel learning algorithm integrating multiple kernel matrices is proposed. The algorithm firstly chooses a part of samples randomly from the whole data set, and then calculates the approximations of the original kernel matrices using the Nystrom algorithm and combines the approximated kernel matrices with the appropriate coefficients accord-ing to the sizes of the approximate errors. The presented algorithm not only reduces the computational complexities but also embodies a directly combination approach. The ex-perimental results for the synthetic and the real-world UCI machine learning databases show that the multi-kernel learning algorithm and the consequent multi-kernel classifiers are quite effective.(2) An explicit multi-kernel mapping algorithm based on the random projection strategy is presented. With the help of the random projection technique, not only can the number of dimensions of attributes be effectively reduce, but also the approximate separability be-tween classes is maintained in the feature space. Using the specific property, we combine the random projection approach with the existing multi-kernel learning algorithm, called the MutliK-MHKS algorithm, to construct the integrated kernel matrices, and thus form the explicit kernel mapping with the partial samples produced by the random selection. Because the number of dimensions of attributes is reduced after the random projection, the computational complexities are thus cut down. At the same time, the approximate separability between classes ensures that the discriminative accuracies of the multi-kernel classifiers are obviously improved for many synthetic and real-world databases.The proposed learning algorithms for multi-kernel classifiers alleviate the contradiction between classification accuracy and computational complexity, to some extent. Compared with the conventional multi-kernel classifiers, the designed multi-kernel classifiers both keep high classification accuracies and reduce the time and space complexities. The advantages of the proposed multi-kernel learning algorithms and the designed multi-kernel classifiers are verified by the theoretical analysis and lots of experimental results.