Classification Methods Based On Clustering And Manifold Regularization

In past years, the classifier design has been one of the important research topicsin the field of pattern recognition, it has made rapid progress in the aspects of theoryand application and a lot of new approaches have been emerging. These newrequirements are often the disadvantages of traditional pattern classification methods,such as classification of massive high-dimensional data, classification under noisejamming and class overlapping, multi-labeled data classification, classification onclass imbalance data sets, optimization of kernel matrices in nonlinear classification,fast non-linear classification, etc. Under this background, this thesis has made furtherresearch on fast robust clustering, classification on class imbalance samples, kerneloptimization and fast semi-supervised classification based on manifold regularization.Several new models and methods have been present to solve the problem of classimbalance, kernel optimization and fast classification.The studies mainly include the following four aspects:(1) To solve the problem of class overlapping and noise jamming, a novelpossibilistic fuzzy clustering algorithm based on the sample-weighted idea wasproposed. In this method, the outliers and the noises have smaller weights, which canlimit the convergence range of typical values. Thus, their contributions to theclustering process are reduced. We proved that its convergence speed was faster thanthat of IPCM (Improved Possibilistic C-Means) algorithm. It can not only reduce thetime complexity effectively, but obtain good clustering accuracy. To solve theclustering problem in linear inseparable case, a robust possibilistic fuzzy kernelclustering was proposed. This method can not only handle linear inseparable and classoverlapping datasets, but also overcome noise interference effectively.(2) The existing class imbalance learning methods can decrease the sensitivity ofSVM to the imbalanced class, but it still suffers from the problem of noises andoutliers. We proposed a new class imbalance method based on the fuzzy and typicalmemberships, which can handle the imbalance problem. Since improved clusteringalgorithm is robust to noises, this method is not only effective on class imbalancedatasets, but also robust to noises and outliers. Experimental results on artificialdatasets and real datasets show that the proposed method is effective for solving theclass imbalance problem, especially for the imbalanced datasets existing noises andoutliers. (3) Most research on Non-Parametric kernel learning (NPKL) has tended tofocus on the semi-supervised scenario. In this paper, we propose a novel unsupervisednon-parametric kernel learning method, which can seamlessly combine the spectralembedding of unlabeled data and manifold Regularized Least-Squares (RLS) to learnnon-parametric kernels efficiently. The proposed algorithm enjoys a closed-formsolution in each iteration, which can be efficiently computed by the Lanczos sparseeigen-decomposition technique. Meanwhile, it can be extended to supervised kernellearning naturally.(4) Compared with traditional computational intelligence techniques such assupport vector machine (SVM), Extreme learning machine (ELM) provides bettergeneralization performance at a much faster learning speed without tuning modelparameters. In order to deal with unlabeled data, we extend the manifoldregularization framework, and demonstrate the relationship between the extended MRframework and ELM. A manifold regularized extreme learning machine is derivedfrom the proposed framework, which maintains the properties of ELM, especially insolving the problem of large-scale data training.This thesis mainly studied three aspects, e.g., classification method for classimbalance samples, classification methods based on non-parametric kerneloptimization and fast semi-supervised classification. Based on the related preparatorywork, several classification and learning models have been established and newalgorithms have been designed. Experiments on benchmark datasets and face datasetsvalidate the effectiveness and efficiency of these proposed algorithms. The results ofthis thesis will enrich the ways to solve the classification problem and has a certaintheoretical significance and good application prospect.