Freature Selection And Extraction Based On Regression And Manifold Learning

In the information era,big data is generated for many fields with big samples and high dimensionality.They have different structures and different categories from variable sources.What's more,valuable information always hides in a mass of invariant information.All of these properties make troubles for their treatment and analysis.Dimensionality reduction is an effective method to solve problems mentioned above.And it has been an independent research direction.One research hotspot in dimensionality reduction is machine learning,which can greatly reduce labor cost by finding inherent relationship automatically on computers.For decades' effort,it has remarkable achievements in dimensionality reduction based on machine learning but many of limitations to be overcome.For instance,intrinsic information could not be used enough by many of algorithms.Besides,the manifold learning method may can't describe a true data manifold accurately.What's more,many methods cannot control the reduced dimensionality flexibly.To address these problems,some improvements and extends are put forward on the basis of state-of-the-art algorithms.Mainly works in this thesis are listed as follows:(1)The state-of-the-art algorithms don't use discriminative information and manifold information simultaneously.This makes inaccurate clustering and/or classification results.To do so,a novel framework for unsupervised feature selection based on kernel fisher discriminant analysis and regression learning(KFDRL)is proposed.The method first constructs a global discriminant objective term of a clustering framework based on kernel method and manifold learning.It then adds another term of regression learning to the objective function imposing the optimization to select a low-dimensional representation of the original dataset.It uses l2,1-norm of the features to impose a sparse structure upon features to be selected.This model could make full use of intrinsic information of original data.Experiments on the method results in better effects on clustering and classification than state-of-the-art algorithms.(2)Kernel method often results a high computing complexity while handling with nonlinear data.Hence,a nonnegative matrix factorization with rank regularization and hard constraint(NMFRC)is researched to overcome it.The NMFRC can also make full use of prior information.For a better description of data manifold,the NMFRC measures similarity of pairwise by geometric distance.In addition,the NMFRC preserves sparsity of data without breaking smoothness of manifold by imposing rank constraint.Labeled information is added in the NMFRC to extend it as semi-supervised.Semi-supervised comparative experiments show that the NMFRC can generate better datasets for clustering.(3)Compared with feature selection algorithms including the KFDRL,the NMFRC with parts-based theory could not control reduced dimensionality arbitrarily.What's more,both of the KFDRL and the NMFRC don't use self-representation and self-similarity of data.Hence,a self-representative feature selection with a NMF framework(SRFS-NMF)is proposed.The algorithm is constructed under a NMF framework and then imposed the self-representation technology to get a low dimensional fitting regression model.An l1-norm constraint is added to be regularization term for feature selection.Thus,the SRFS-NMF can not only uses the advantages of parts-based theory from NMF and properties of self-representation of data,but also achieves arbitrary number of selected features.Experiments show that the SRFS-NMF performs better than both NMF-based algorithms and norm regularizationbased feature selection algorithms.