Classification Algorithm Based On Multi-view Discriminant Feature Learning

Pattern recognition, machine learning and other cross disciplinary need to discover the rules from observed data. With the revolutionary development of Internet, communication and other information technology in recent ten years, the amount of data nowadays is growing at rapid rate. Some of the data can be expressed in a variety of forms. For example, in the Internet, each web page can be represented as the documents in the web and hyperlinks pointing to it; in the field of face recognition, the features extracted from one face such as Gabor feature, HGG feature, LBP feature, PCA feature can be used to correspondly describe the orientation feature, edge feature, local variation of pixel features and major informative features. The traditional related algorithms are based on single view which only uses structural characteristics in one view, without using correlation and complementary information between different views. Multi-view learning approaches attempt to extract associated and complementary characteristics between the different views, and can improve the classification effect. Therefore, in recent years, multi-view feature learning in machine learning, data mining and computer vision has been widely studied. This paper focuses on the topic of multi-view data and elaborated in three aspects with discriminant feature abstraction in subspace, semi-supervised learning and discriminant dictionary learning. The main innovative work include following points:(1) By focusing on the analysis of DCCA(Discriminant Canonical Correlation Analysis), Mv DA(Multi-view Discriminant Analysis) and CECCA(Combined Enhanced Canonical Correlation Analysis), this thesis proposes the DDCA(Dual Discriminant Correlation Analysis) method. DDCA model has two advantages: one is that in each single view Fisher discriminant analysis(FDA) is used to make samples separable; the other one is that the correlation between different views is analyzed, that is to find the projection vectors to maximize the correlation in same class samples and minimize correlation between different classes. DDCA is a supervised feature extraction method and can effectively utilize the label information of samples compared with CCA. In addition, the traditional correlation analysis method only can be used in different view and ignore the same view information as the model limitations. DDCA can analyze the correlation both in same view and different views and is helpful to improve the classification result.(2) In semi-supervised learning, feature extraction of samples in multi-view is helpful to further analyze the sample information. Researchers have proposed some effective semi-supervised multi-view learning approaches and achieved some good results, but the different views and same view information are not well considered in these methods. In addition, how to effectively extract neighborhood structure information for labeled and unlabeled samples also has great potential. This thesis proposes a semi-supervised multi-view feature learning method called SDvFL(Semi-supervised Dual-view Feature Learning). This method can make the labeled and different class samples in the same view separated, the unlabeled and non-neighbor samples in the same view closed; meanwhile make the labeled and different class samples in different views separated, the unlabeled and non-neighbor samples in different views closed.(3) Based on the research about relationship between different views in semi-supervised condition, this thesis further introduces the concept of view consistency to analyze the association between the projection matrix from different views, and proposes the SDvCFL(Semi-supervised Dual-view Consistency Feature Learning). SDvCFL considers the features in multi-view describe the characteristics of different aspects for the same object. Thus the projection matrixes from different views should also have certain relation. Therefore the structure information of the different views should be similar, which is called the "view consistency" in this thesis. SDvCFL utilizes the view consistency to further add constraints on the difference of the structure information in original sample.(4) Sparse representation and dictionary learning technology have been widely stuied. Thus this thesis proposed a discriminant dictionary learning method named MDDL(Multi-view Discriminant Dictionary Learning) for the multi-view data based on the traditional single view dictionary learning. MDDL model is capable of learning a structured discriminant dictionary with three advantages: firstly, the samples of same class can use corresponding class dictionary in the same view to approximately represent; secondly, samples represent by different class in all the views have large residuals; thirdly, the coefficients discriminant terms are introduced to further strengthen the discrimination ability of the dictionary.(5) Furthermore, this thesis analyzes the properties of sparse reconstruction coefficients based on point(4). In supervised condition, this thesis redesigns the reconstruction coefficients to make the different class and labeled coefficients in the same view separated, the unlabeled and non-neighbor coefficients in the same view closed; meanwhile it can make the different class and labeled coefficients in different views separated, and the unlabeled and non-neighbor coefficients in different views closed. The new model can be called NMDDL(Neighbor Multi-view Discriminant Dictionary Learning). NMDDL can further improve the discrimination based on retaining the neighbor relation and NMDDL improves the classification result.