Research On Clustering Task-driven Unsupervised Feature Learning Methods

Unsupervised feature learning is one of the most fundamental unsupervised tasks in machine learning.It aims to learn meaningful representations from massive unlabelled data.In the scenario of high dimension data,the performance of traditional data analysis algorithms deteriorates notably.The efforts have been made to counter this issue by applying feature learning methods as a pre-processing step.It has been widely used in classification,visualization and pattern recognition,etc.During the past decades,the unsupervised feature learning task has attracted lots of attention and made great development due to its importance.However,it still suffers from the major drawback:feature learning methods only use the reconstruction as supervised information.Therefore,features from these methods lack discriminative information.Essentially,features should both contain the essential and discriminative information.In this thesis,the novel feature learning methods are proposed which incorporate the core idea of clustering tasks into deep networks to tackle the above issue.These methods can contain essential information and discriminative information.In summary,this thesis focuses on the clustering-driven unsupervised feature learning and aims to improve the discriminability of representations.The main content and novelty of this thesis are summarized as follows:(1)For the lack of discriminative information and the approximation error of reconstruction information in Deep Semi-NMF(DSN),Discriminative Deep Semi-NMF network(DDSNnet)is proposed which is motivated by the discriminative idea in clustering.This method can obtain the non-negative features and have no approximation errors.In addition,the novel similarity measurement is proposed,which includes global similarity and local similarity between the original data and features.It can encourage the compactness between similar points and separateness between dissimilar points in feature space.Furthermore,the experiments validate the proposed assumptions from the visualization and the clustering performance.(2)For the lack of discriminative information of features,Clustering with Orthogonal Auto Encoder(COAE)is proposed which is motivated by the spectral clustering and deep clustering methods.It can encourage the orthogonality of the learning features in autoencoder according to an orthogonality regularization term.We consider that the orthogonality encourages more diversity in latent features and enlarges the differences in the between-class latent features.For the limited constraint of the regularization term,COAE enhances the discriminative information via joint optimization of clustering tasks and feature learning.The framework based on the classification task idea to make the predicted labels and auxiliary labels more deterministic.Therefore,COAE can learn discriminative information that compacts between similar points and separates between dissimilar points in feature space.Furthermore,the experiments validate the proposed assumptions from the visualization and the clustering performance.(3)For the need to handcrafted auxiliary labels and less reliable supervised information in the classification task framework,Fuzzy clustering and extended Mutual information with Auto Encoder(FMAE)is proposed which is motivated by the fuzzy clustering and mutual information.It adopts the weighted intra-class variance as the objective function to achieve the joint optimization without any auxiliary labels and closer features in the same category.In addition,the extended mutual information can use the original data as supervised information,which further enhances the reliability of the predicted labels as discriminative guidance information.Furthermore,the experiments validate the proposed assumptions from the visualization and the clustering performance.The thesis focuses on unsupervised deep networks of feature learning methods.In terms of the limitations of existing methods,this thesis proposes clustering task-driven methods to improve quality of features which can provide powerful support for other data analysis tasks.