Discriminant Feature Learning Based On Stochastic Neighbor Embedding

The purpose of feature learning is to obtain effective representation of the raw data,and then improve the performance of machine learning algorithms such as clustering or classification.Some of the existing feature learning algorithms focus on maintaining the topological structure of the raw data in the process of feature learning,but ignore the discriminant information in the raw data.Topological structure,as an important descriptive information of the raw data,can be used to guide feature learning,but it cannot fully reflect all the information contained in the raw data.Discriminant information,as another description of the raw data,is even more important than topological structure in some clustering and classification tasks.Therefore,the feature representation obtained by the fusion of topological structure and discriminant information will have better expression ability.In order to learn topological structure and discriminant information in the raw data simultaneously,this thesis proposes Discriminant Feature Learning based on t-distribution Stochastic Neighbor Embedding(DTSNE).The proposed model organically integrates the objective function of clustering algorithm and stochastic neighbor embedding algorithm,and then uses gradient descent with momentum method to update parameters of two types,which makes sample points in the mapping space reflect both topological structure and discriminant information in the raw data.Compared with stochastic neighbor embedding algorithm,DTSNE can generate discriminant representation of the raw data,as a result,the expression ability of the data feature is improved.In order to further enhance the discrimination of the data feature representation,this thesis draws on the idea of semi-supervised ensemble learning and proposes Discriminant Feature Learning based on t-distribution Stochastic Neighbor Embedding guided by Pairwise Constraints(pc DTSNE).pc DTSNE introduces pairwise constraints by cluster ensemble and uses these pairwise constraints to impose penalties on the objective function,which makes sample points in the mapping space present stronger discrimination.Compared with DTSNE,pc DTSNE’s learning ability of discriminant information in the raw data has been further strengthened,as a result,the expression ability of data feature representation has also been significantly improved.In order to verify the feature learning performance of DTSNE and pc DTSNE,extensive experiments are carried out on several public data sets.The experimental results show that DTSNE has better performance in both clustering and classification tasks.With the introduction of pairwise constraints,the expression ability of data representation generated by pc DTSNE is further improved.