Sparse Feature Learning:Algorithms And Applications

Feature learning is a key factor in the success of machine learning algorithms,since a good feature representation can mine the potential statistical property of natural data effectively.Physiological researches in neuroscience suggest that the receptive field of V1 population responses to natural stimuli may be a result of sparse representation(SR).From them on,sparse feature learning has attracted much attention and study from researchers.In the field of sparse feature learning,there are two mainly technologies: the design of sparse coding(SC)algorithms for reconstructing sparse features and the extension of SR models for specific applications.Based on the research of the theories and technologies of sparse feature learning,this dissertation focuses on designing more effective SC algorithm for its mathematical model-norm regularized least squares problem,and extending this model for image denoising and feature selection applications.The main works and innovation are as follows:(1)A homotopy coordinate descent optimization algorithm is proposed for sparse feature reconstruction.The existing optimization algorithms for solving the-norm regularized least squares problem prone to producing large reconstruction error and low computational efficiency.To this end,a new sparse coding algorithm is proposed which combines the coordinate descent method with the homotopy strategy.Meanwhile,in order to reduce the computational time,two strategies are imposed to the coordinate descent method: active set updating and a strong rule for active set initialization.Experimental results show that this proposed algorithm can achieve better balance between reconstruction performance and computational time.(2)A joint-norms minimization model for random sparse noise removal is proposed.Traditional SR models use-norm to define loss function,which cannot produce sparse error that is not suitable for sparse noise removal.To this end,a SR model based on-norm loss term is proposed for random sparse noise denoising.Meanwhile,in order to be able to solve this problem,a joint-norm convex model is proposed for approximation.Experimental results are reported to compare with some state-of-the-art SR based methods and demonstrate that the proposed method can preserve more details and texture,and improve the denoising performance for random sparse noise removal.What’s more,this proposed method can be used to re-denoise random-valued impulse noise(RVIN),which can further improve the denoising performance.(3)A-norm regularized minimization model for supervised multi-class feature selection is proposed.The traditional-norm regularized least squares model is only suitable for two-class feature selection task,which has less efficient in multi-class feature selection task.To this end,a-norm regularized structured SR model is proposed for multi-class feature selection,and an efficient matrix homotopy iterative hard thresholding(MHIHT)algorithm is derived to solve the above optimization problem.Furthermore,in order to reduce the computational time,an acceleration version of MHIHT(AMHIHT)is derived.Experiment results show that the proposed approach keeps a good balance between recognition accuracy and computational efficiency.(4)A nonnegative spectral analysis with-norm regularization unsupervised feature selection method is proposed.For unsupervised feature selection task,traditional methods based on nonnegative spectral analysis and-norm equality constraint structured SR model need to speed much time adjusting the number of selected features.What’s more,these methods use a fixed similarity matrix,which is unstable to noise.In order to address these problems,a novel structured sparsity-based method that incorporates nonnegative spectral analysis with a-norm regularization term is proposed.Furthermore,a graph regularization term is imposed to learn the similarity matrix adaptively.A simple and efficient alternative algorithm is designed to solve the proposed optimization problem with the analysis of computational complexity.Experimental results prove that the proposed method has the advantage of automatically determining the number of selected features compared against the state-of-the-arts,and the selected features have better clustering performance.