Research On Multiple Kernel Learning Algorithms And Applications

Kernel learning algorithms have been widely used in computer vision,natural language processing,remote sensing image analysis and many other fields.Kernel learning implicitly projects the original data to a highly dimensional space(even infinitely dimensional space),and it changes the linear nonseparable data in the original space into the linear separable data in the kernel space by kernel trick.The main task of single kernel learning is how to select a suitable kernel parameter for data expression in order to improve the performance of kernel learning.But single kernel learning can not deal with data with multi-source and heterogeneous characteristics.Multiple kernel learning can skillfully use the data with multi-source and heterogeneous characteristics by fusing a group of base kernels.Compared with single kernel learning,multiple kernel learning uses a combined kernel to replace single kernel,and transforms the problem of kernel selection faced by single kernel learning into the problem of optimizing the base kernel combination coefficient.Therefore,in multiple kernel learning,it is a very important task to choose or optimize the learning objective and the base kernel combination coefficient.The contribution of this paper mainly includes the following four aspects:(1)We propose a fast multiple kernel clustering algorithm based on self-weighted local kernel alignment.Although the algorithm based on the local kernel alignment multiple kernel clustering framework has achieved good clustering performance,there are two defects in the clustering framework.1.Ignoring the differences between local kernels of samples,namely using each local kernel simply and equally in the clustering process,thus failing to make full use of the contribution of each local kernel alignment to the clustering performance.2.The time complexity of the clustering framework is too high,and the clustering time is highly dependent on the number of local kernels involved in clustering.At the same time,the clustering framework ignores the similarity between local kernels,resulting in a lot of redundant local kernels participating in the clustering.In order to solve the first problem,we introduce a new optimization variable(local kernel contribution weight)to measure the contribution of each local kernel alignment to the clustering performance,and simultaneously optimizing contribution weight,kernel combination coefficient and clustering relationship matrix.Then,we design a three-step alternative optimization algorithm to solve the new optimization problem.In order to solve the second problem,we use anchor technology to reduce the number of local kernel involved in clustering,which greatly reduces the clustering time of the local kernel alignment based algorithm.The specific idea of anchor point is: the cluster centers of samples are regarded as the anchor point samples,only local kernel alignment is performed on anchor point samples,and the number of anchor points can be automatically determined by using the expanding local kernel matrix to reconstruct the global kernel matrix.A large number of experiments prove the clustering performance and time performance of the proposed algorithms.(2)We propose a multiple kernel clustering algorithm framework based on global and local structure alignment.In order to solve the problem that the existing multiple kernel clustering algorithms often ignore the local structure of samples in the process of clustering,we propose a multiple kernel clustering framework based on global structure and local structure alignment.The local structure of samples is described by the local form in the local kernel alignment clustering framework,while the local structure of samples is mined by the global manner in the multiple kernel clustering framework based on global structure and local structure alignment.The proposed clustering framework can capture the local structure of samples from a global perspective by using manifold learning algorithms.To illustrate the generality of the proposed framework,we instantiate two concrete multiple kernel clustering algorithms based on global structure and local structure alignment.One is to use local linear embedding to code the local structure of samples;the other is to use local preserving projection to capture the local structure of data.A large number of experimental results show the effectiveness and generality of the proposed multiple kernel clustering framework.(3)We propose a multiple kernel extreme learning machine framework based on kernel interaction.In order to solve the problem that the existing algorithms under the framework of multiple kernel extreme learning machine only use the information of the corresponding single kernel while neglecting the interaction between kernels during the process of kernel fusion,we propose two improved algorithms.One is called multiple kernel extreme learning machine based on matrix induced regularization.That is to say,a positive definite matrix is designed to describe the correlation between base kernels,and then the regularization term induced by the matrix is used to constrain the optimization of the base kernel combination coefficient.The algorithm can make good use of the prior knowledge from data to improve the performance of multiple kernel extreme learning machine.The other is called multiple kernel extreme learning machine based on matrix r,p norm.In this algorithm,the r,p norm of the coefficient matrix is used to replace the p norm of the coefficient vector to control the optimization of the base kernel combination coefficient.Compared with multiple kernel extreme learning machine based on p norm,multiple kernel extreme learning machine based on matrix r,p norm has more flexibility in the selection of base kernel combination coefficient.Then,the corresponding optimization methods have been designed for two proposed algorithms,and their convergence has been proved.The experimental results show that the generalization performance of the proposed multiple kernel extreme learning machine is better than that of the traditional multiple kernel extreme learning machine.(4)We propose a multiple kernel extreme learning machine based on kernel alignment regularization.In order to optimize the base kernel combination coefficient,the label information of the samples is ignored in one-stage multiple kernel extreme learning machine,and in two-stage multiple kernel extreme learning machine,although the label information of the samples is used,the interaction between the optimization process of the base kernel combination coefficient and the optimization process of the network structure is ignored.To address these issues,a one-stage multiple kernel extreme learning machine based on kernel alignment regularization is proposed.The specific idea is considering the sample label matrix as the ideal kernel in the kernel alignment criterion,and then adding the criterion as a regularization term to the optimization goal of multiple kernel extreme learning machine.Then,the corresponding optimization method has been designed and the convergence of the algorithm has been proved.The experimental results show that the proposed multiple kernel extreme learning machine based on kernel alignment regularization has good classification performance.