Research On Multiple Kernel Learning Algorithms And Their Applications

Kernel methods are important machine learning methods and fit for complex classification, feature extraction and other nonlinear pattern analysis tasks. The performance of kernel methods relies heavily on the chosen kernel function, therefore, the multiple kernel learning(MKL) methods have been presented. MKL adaptively learns an optimal combined kernel from a group of basic kernels, which can well overcome the kernel selection issue. More importantly, MKL can be regarded as the type of kernel level fusion between the feature level and the decision level fusion of multiple information sources. Thus, the MKL algorithms have great practical significance and value. Although the MKL algorithms are mature now, it is still needed to explore more efficient solutions, combine the idea of MKL with more research directions of machine learning, and apply the MKL algorithms to more real world application fields. Consequently, this dissertation mainly focuses on designing efficient two stage MKL algorithms and applying them to radar emitter recognition and hyperspectral image classification tasks. Meanwhile, the distance metric learning(DML) problem is also investigated. The main contributions of this dissertation are as follows:Firstly, using the strategies of grouping and combining, an algorithmic framework of two stage multiple kernel canonical correlation analysis(TSMKCCA) is proposed. In the first stage of TSMKCCA, the basic kernels built on each feature representation are divided into two groups, and then a combined kernel is calculated for each group according to the uniform weighting scheme or kernel target alignment(KTA) based heuristic weighting scheme. In the second stage of TSMKCCA, the existing kernel canonical correlation analysis(KCCA) algorithm or supervised KCCA algorithm is performed for final nonlinear feature extraction. The TSMKCCA can not only rapidly extract nonlinear correlated features from multiple feature representations, but also integrate the way of feature level fusion and kernel level fusion, which brings excellent recognition performance. The experimental results of handwritten numeral and face data illustrate the efficacy of our proposed methods.Secondly, by employing the information fusion function of MKL, two approaches based on the multiple kernel support vector machine(SVM) algorithms and the algorithmic framework of TSMKCCA are proposed respectively for specific radar emitter identification. Thus, the application fields of MKL is furtherly expanded. For feature representations of radar emitter signals in different domains, the corresponding basic kernels are constructed firstly, and then we use multiple kernel SVM or TSMKCCA to fuse multiple features for final identification of different emitters. Since the Doppler-slice of ambiguity function of radar emitter signals has complementary information, the proposed methods can effectively fuse them, obtaining excellent identification performance. The experimental results of real radar data demonstrate the validity of the proposed methods.Thirdly, to obtain efficient multiple kernel extreme learning machine(MKELM), the two stage MKELM algorithms based on empirical composition and Fisher discriminant ratio are proposed respectively and applied to the hyperspectral image classification task. The empirical composite MKELM(EC-MKELM) algorithm determines the kernel weights totally by the experts’ experience, while the Fisher ratio-based MKELM(FR-MKELM) algorithm calculates the kernel weights adaptively. Moreover, a modified extended morphological profile(m EMP) feature is proposed to extract the spatial feature of hyperspectral images. The proposed EC-MKELM algorithm and FR-MKELM algorithm can fuse the spectral feature、Gabor feature and m EMP feature effectively for spectral-spatial classification of hyperspectral images. The experimental results of real hyperspectral images demonstrate the performance merits of the proposed feature and MKELM algorithms.Finally, for the existing constrained empirical risk minimization based DML, it succeeds theoretically and algorithmically, but is not applicable to the popular non-differential hinge loss function. To remedy this deficiency, we resort to the smoothing approximation method and thus propose the algorithm of hinge loss-based metric learning via smoothing(HLMLsm). HLMLsm converts the constrained empirical risk minimization problem built on the hinge loss to an equivalent non-smooth convex optimization problem with the special min-max form, which can be directly solved by the Nesterov’s smoothing approximation method. Moreover, HLMLsm uses line search to automatically determine the Lipschitz constant, accelerating the convergence rate. Besides, the local triple training data are used to preserve the geometric structure of data and save the computational time. The experimental results of UCI benchmark data demonstrate the effectiveness of the proposed method.