Research On Multi-view Clustering Algorithms

As one of the most important machine learning techniques,kernel methods provide a powerful and unified learning framework,which allows researchers to focus on developing efficient learning algorithms,regardless of specific data types such as string,vector,text,graph,etc.Due to the above merits,kernel methods have been intensively applied into different learning tasks,including classification,regression,clustering,ranking and so on.As well known,the performance of kernel methods is critically dependent on the choice of kernels and their parameters.However,how to choose an appropriate kernel and the corresponding parameters is an open issue and lacks principled guidelines.Consequently,the research on kernel methods is worth exploring and of critical importance in applications.The work in this thesis treats each base kernel as one view,and aims to address the above issues by developing several effective multi-view clustering methods.Its main contributions are summarized as follows.1.We propose an MVKM clustering with a novel,effective matrix-induced regularization to reduce such redundancy and enhance the diversity of the selected kernels.We theoretically justify this matrix-induced regularization by revealing its connection with the commonly used kernel alignment criterion.Furthermore,this justification shows that maximizing the kernel alignment for clustering can be viewed as a special case of our approach and indicates the extendability of the proposed matrix-induced regularization for designing better clustering algorithms.As experimentally demonstrated on five challenging MKL benchmark data sets,our algorithm significantly improves existing MKKM and consistently outperforms the state-of-the-art ones in the literature,verifying the effectiveness and advantages of incorporating the proposed matrix-induced regularization.2.We propose an optimal neighborhood kernel clustering(ONKC)algorithm to enhance the representability of the optimal kernel and strengthen the nego-tiation between kernel learning and clustering.We theoretically justify this ONKC by revealing its connection with existing MKKM algorithms.Furthermore,this justification shows that existing MKKM algorithms can be viewed as a special case of our approach and indicates the extendability of the proposed ONKC for designing better clustering algorithms.An efficient algorithm with proved convergence is designed to solve the resultant optimization problem.Extensive experiments have been conducted to evaluate the clustering performance of the proposed algorithm.As demonstrated,our algorithm significantly outperforms the state-of-the-art ones in the literature,verifying the effectiveness and advantages of ONKC.3.We propose a simple while effective algorithm to address multi-view clustering with incomplete views.Different from existing approaches where incomplete kernels are firstly imputed and a standard MKC algorithm is applied to the imputed kernels,our algorithm integrates imputation and clustering into a unified learning procedure.Specifically,we perform multiple kernel clustering directly with the presence of incomplete kernels,which are treated as auxiliary variables to be jointly optimized.Our algorithm does not require that there be at least one complete base kernel over all the samples.Also,it adaptively imputes incomplete kernels and combines them to best serve clustering.A three-step iterative algorithm with proved convergence is designed to solve the resultant optimization problem.Extensive experiments are conducted on four benchmark data sets to compare the proposed algorithm with existing imputation-based methods.Our algorithm consistently achieves superior performance and the improvement becomes more significant with increasing missing ratio,verifying the effectiveness and advantages of the proposed joint imputation and clustering.4.We propose a novel localized incomplete multi-view k-means algorithm,imposed with matrix-induced regularization.Besides utilizing such matrixinduced regularization to handle the correlation of base kernels,our algorithm only enforces the similarity of a sample to its k-nearest neighbors to sufficiently align with their ideal similarity values.A three-step iterative algorithm with proved convergence is developed to solve the resultant optimization problem.Comprehensive experiments are conducted on eight benchmark data sets.As indicated,our algorithm significantly outperforms the state-of-the-art comparable algorithms proposed in the recent literature,verifying the benefits of maximizing localized kernel alignment and incorporating the matrix-induced regularization.