The Study Of Kernel Methods In Classification, Regression And Clustering With Applications

Recent years, kernel method develops rapidly in pattern recognition and machine learning community. The nature of kernel method is to map data from low dimensional input space to high dimensional feature space, which can improve the performance of machine learning method. For example, for non-linearly separatable dataset in input space, the mapping may make it linearly separatable in feature space. In kernel method, there exist important problems to be solved. Among them, robust support vector regression, semi-supervised multi-label learning, sparse support vector learning and kernel clustering are in need of solutions.In this dissertation, these four problems are investigated. The contributions of this dissertation are as follows:Firstly, we propose an adaptive error penalization support vector regression method named AEPSVR. AEPSVR can reduce the affect of outliers, and achieves improved generalization capability. Furthermore, we investigate the properties of cost function for constructing robust support vector regression. Then a family of robust cost functions is introduced. Based on these cost functions, we implement a fuzzy robust support vector regression method called FRSVR, which is robust, and can be used to identify outliers.Secondly, for semi-supervised multi-label support vector learning problem, we present a semi-supervised multi-label learning method named SSML_SVM to obtain an effective multi-label method for gene expression data processing. The proposed SSML_SVM transforms semi-supervised multi-label learning into semi-supervised single-label learning by PT4 method, then it labels unlabeled examples using MAP (Maximum a Posteriori) principle together with K-nearest neighbor method, and finally, it solves single-label learning problem using SVM. The distinctive character of the proposed method is its efficient integration of SVM based single-label learning together with MAP and K-nearest neighbor method.Thirdly, we extend direct sparse kernel learning framework to support vector regression, and propose direct sparse kernel regression method called DSKR. By adding a non-convex constraint toε-SVR, DSKR can obtain sparse kernel regression with arbitrary user-defined number of support vectors. It can achieve promising regression performance with less number of support vectors thanε-SVR.In the last, we propose two improved kernel affinity propagation clustering methods called SSKAPC and AFAPC. Kernel trick is adopted for the purpose of processing non-linear problem. In SSKAPC, affinity propagation clustering method is extended to semi-supervised setting, in which background knowledge is provided in terms of pairwise constraints for improving clustering performance. In AFAPC, clusters and corresponding centers can be achieved by transforming affinity messages in data networks, where affinity messages are obtained based on gravity forces between data points. Experimental results demonstrate that the clustering accuracy of AFAPC is comparable with affinity propagation clustering. However, its running time is much less than that of affinity propagation clustering method.The author also does researching work on image forensics, and proposes a fake image detecting method named BERFS. BERFS can identify fake images using relative frequency feature and semantic feature with high accuracy, and can estimate blurred region precisely.