Research And Application Of Manifold Regularization Multiple Kernel Model On Supervised And Semi-supervised Classification

Data classification is one of most basic learning tasks in machine learning. With the development of informatics and networks, the complexity of the data required for classification is increasingly high. Multiple kernel learning, due to the strong ability of describing characteristics of the data, is an effective method of complex datasets classification. From the view of classification, one dataset is divided into two parts:input data, belonging to space or attribution information of dataset and the corresponding output data, belonging to class label information of dataset. Since the input data from the natural world or engineering, they are often inherent constrained or have constrained relationships that can be described by mathematical manifold. Manifold constraints of input data in its own space, possess intrinsic characteristic. Further, it is important information for people to identify targets. However, multiple kernel classification method has not been fully utilized the information.To take advantage of manifold constraint information of the input data sample, this thesis presents a kind of supervised manifold regularization multiple kernel model. To obtain the manifold constraint information of the input data samples, it is necessary to describe the degree of neighborhood relations in their space. This thesis applies Hellinger distance which can evaluate the neighbor relationship carefully; at the same time, considering the effect of label information expressed by the output data, that is to say the neighbor relations of the same class have a higher degree than that of the different class. Finally, supervised manifold constraint regularization term with consideration of label information is proposed, which is incorporated into multiple classification model. The supervised manifold regularization multiple kernel model is established and the algorithm of the model is given. Results of the supervised classification simulation comparisons show that the proposed supervised manifold regularization multiple kernel classification model is effective.For the actual engineering background, the output data are combined with both labeled and unlabeled data. This thesis will extend the supervised manifold regularization multiple classification model to a kind of semi-supervised model. First, to get the neighbor relationship by Euclidean distance among all input data samples, and thus obtain the manifold constraint information of the input data samples; then, to extend the matrix of the multiple kernel functions of all input data samples and calculate manifold regularization by the manifold constraint information of all input data samples; thus, the model is extended to a semi-supervised manifold regularization multiple classification model which can utilize all the data samples. This thesis gives the algorithm, error analysis and simulation comparison of the semi-supervised manifold regularization multiple kernel classification model. Simulation results show the effectiveness of the proposed semi-supervised model.Specific to the semi-supervised manifold regularization multiple kernel model, on the one hand, to improve the adaptability and classification accuracy of the model, this thesis presents a kind of automatic selection method of multiple kernel parameters; on the other hand, to improve the flexibility of the model, the thesis improves the multiple kernel combination weights constraint form. As for the parameter automatic selection aspect, this thesis improves the mathematical expression of the semi-supervised manifold regularization multiple kernel model and designs the algorithm to transform the kernel parameter values into the solutions of the algorithm. As for the constraint form of the multiple kernel combination weights, this thesis improves the fixed 1 norm constraint to the general p-norm constraint, and gives the solving theorems of semi-supervised manifold regularization multiple kernel model with the p-norm constrained multiple combination weights and presents the proofs to achieve the improvement of the model flexibility. For the improved semi-supervised classification model by automatic selection of kernel parameters and p-norm constrained multiple combination weights, this thesis presents the simulation comparison tests. The results show that the proposed semi-supervised manifold regularization multiple kernel model with kernel parameters automatically selection and with p-norm constrained multiple combination weights are both effective.