Statistical Sparse Learning:Feature Extraction,Clustering, Classification And Multi-feature Fusion

As a cross-disciplinary subject between statistics and artificial intelligence, statistical learning has developed rapidly in recent years. Statistical learning has been used in vari-ous fields, such as data mining, machine learning, information retrieval, computer vision, etc. Among this subject, the statistical methods based on sparse theory has been widely applied with the help of extensive research and application of sparse decomposition and compressed sensing. These methods attract a great deal of attention from researchers, and have a far-reaching impact on the development of the subjects of information science and statistics. Currently, researchers from statistics have begun to extend classic learning theo-ries and methods by considering some sparse properties, and have been establishing a new subject about statistical sparse learning.Focusing on statistical sparse learning, in this thesis, we conduct some in-depth and extensive research on its theories and applications on dimensionality reduction, feature extraction, transfer learning, clustering, classification and so on, and propose some new ideas on these fields. The main content of this thesis is listed as below:1. We first introduce the background and significance of statistical sparse learning, and explore its applications on visual problems. Then, we survey its current research and developments at home and abroad.2. We investigate some sparse principal component analysis methods proposed in re-cent years, and analyze their limitations and drawbacks, especially of which these methods cannot handle high-order data directly but have to vectorize these data in advance. As we know, this vectorization process destroys the valuable spatial re-lationship between data. Focusing on this, we propose a sparse high-order princi-pal component analysis algorithm to deal with the high-order tensorial data directly. Through our algorithm, we can do dimensionality reduction and feature extraction directly on high-order tensorial data.3. In clustering task, we begin our topic by analyzing the widely-used k-means algorith-m and a newly-proposed method based on dictionary learning framework, and point out their limitations and drawbacks. Concentrating on these weak points, we propose a new clustering method based on sparse representation and dictionary learning. Our method learns a dictionary for each latent class, rather than learning a centroid point for the class like k-means. Then, our algorithm learns a common dictionary that is shared by all the latent classes. This common dictionary captures the common infor-mation that are not helpful for clustering but are essential for representation of the data. This method works in a multi-task learning fashion, and improves clustering performance to a visible extent.4. In transfer learning area, we investigate a challenging task that research have not touched yet on transferring unlabeled and heterogenous data unsupervisedly. Facing this problem, we propose a new unsupervised transfer learning algorithm based on sparse representation and dictionary learning. Our algorithm tries to find a projection space for the source domain, and drives the projected data from the source domain to resemble those from the target domain. Then, our method selectively choose the more informative data to transfer, by which way, the task of the target domain can be more effectively resolved.5. For image classification task, we investigate recent dictionary learning based algo-rithms deeply and extensively, and show their shortcomings. Focusing on these draw-backs, we propose a novel dictionary learning based image classification approach. Our method learns a discriminative dictionary for each class, and learn one common dictionary for all the classes. This common dictionary captures the essential informa-tion used for reconstructing the image data, but this kind of information do not help classification. Through our method, we can learn a more compact and more discrimi- native dictionary for classification. More importantly, we can separate the distinctive information and the shared information within one class.6. We extend the our classification method further to deal with multiple features. As we know, compared with image classification on a single feature, multiple features of one image can do much good to classification of them. Therefore, based on sparse representation and dictionary learning, we propose a classification-oriented multi-ple features fusion algorithm. Our method not only deals with multiple features for classification, but also can fuse them into a more compact and more discriminative new features to represent an image. As a result, the classification performance can be enhanced.7. Finally, we summarize this thesis, and illustrate some directions worth exploring and researching on.