Multi-label Dimensionality Reduction Algorithm Research Based On Multiple Kernel Learning

In multi-label classification, one instance is probably denoted by a single label or multiple labels. Multi-label datasets generally contain irrelevant, redundant and noise features, which is more likely to influence the performances of classification and leads to high computational complexity. Therefore, dimensionality reduction is an important pre-processing in multi-label classification.It is found out that the combination of kernel learning and classifier design can improve the classification performance. This greatly promotes to apply kernel to other areas, for example dimensionality reduction. When kernel learning is combined with dimensionality reduction, the performance lies on the kernel mapping. Therefore selecting a proper kernel type and its parameters becomes a new problem. In this thesis, we deal with this problem using multiple kernels learning (MKL) and propose two dimensionality reduction algorithms, i.e., multiple kernel principle component analysis (MK-PCA) and multiple kernel multi-label linear discriminant analysis(MK-MLDA).For MK-PCA, this algorithm is based on the conventional unsupervised dimensionality reduction algorithm PCA and MKL. This thesis finds that kernel combination coefficients with 1-norm constraint cannot obtain the optimal solution.Therefore, a combination coefficient with 2-norm constraint is adopted. When RBF kernel is used, the solution of MK-PCA is the equal kernel matrix weight coefficient,namely average kernel (Av-KPCA). In order to make full use of MKL, this thesis adds the entropy of information to multiple kernels PCA. The experiments show that MK-PCA with regularization and Av-KPCA obtain comparable and more performance by comparison with single kernel model under the condition of no need to adjust the kernel parameter.For MK-MLDA, this algorithm is based on the traditional supervised dimensionality reduction algorithm MLDA and MKL. Linear discriminant analysis(LDA) is one of the widely-used single-label supervised dimensionality reduction techniques, which has been extended to multi-label case, i.e., MLDA. Due to the prominent performance of kernel, MLDA based the kernel learning is divided into Kernel MLDA (KMLDA) and Multiple kernel MLDA (MK-MLDA). Further we optimize projection matrix and base matrix coefficient using alternative way. In the experiments, the proposed MK-MLDA model is superior to the other algorithms in the most performance indexes.To summarize, MKL is added to dimensionality reduction algorithm, which indeed improves the performance of classification and there is a benefit to avoid the problem of selecting the proper kernel parameter. The experimental results correspond well to the theoretically calculated ones.