Joint Sparse Projection For Image Feature Extraction

Dimensionality reduction is one of the basic task of machine learning.Dimensionality reduction aims to extract or select critical features from the high-dimensional original spaceand solve the curse of dimensionality caused by high-dimensional features.The representative traditional feature extraction methods are principal component analysis?PCA?and linear discriminant analysis?LDA?.Since they are based on the global structure of data to extract low-dimensional features,when the high-dimensional data distribution obviously has the local geometry property or approximate manifold structure,the performance of the traditional feature extraction methods are usually poor.Although the feature extraction methods based on manifold learning can preserve the local geometry of high-dimensional space and have better performances than the tradictional methods,there are still many shortcomings.Through analyzing and comparing the advantages and disadvantages of these methods,this paper summarizes three problems.First,some features extraction methods?i.e.LDA?exist the small-class problems,which mean that the number of the learned projections is limited by the number of classes.Second,these methods begin to introduce sparse constraints into manifold learning based feature extraction by usingL₁-norm penalty on the regularization term,the learned projection can be sparse.The methods based on the ₁L-norm penalty term does not provide a jointly sparsity.Third,some methods use the ₂L-norm to measure the loss on the objective function,which causes the algorithms to be sensitive to the outliers,while the real images are corrupted by noise.Therefore,it's important for us to pay more attention to the robustness of the algorithm.To solve the three problems summarized above,based on manifold learning and joint sparse learning,three new algorithms are proposed in this paper,namely Joint Sparse Local Preserve Projection?JSLPP?,Joint Sparse Neighbor Preserve Projection?JSNPP?and Deep Learning feature based Joint Sparse Local Preserve Projection?DL-JSLPP?.For the problem1,the proposed news methods avoid the small-class problem by designing a novel objective functions;for problem 2 and problem 3,the proposed methods based on manifold learning can obtain the optional projection with joint sparsity,which can simultaneously perform feature extraction and selection.At the same time,the jointly sparse projections have stronger semantic interpretation at the feature level.The three algorithms are closely related in theory and they are progressively.Firstly,the features extracted by the three algorithms can preserve manifold structure or local geometrical structure of the original data.They preserve the local structure of the original data in different ways.JSLPP and DL-JSLPP preserve the data point's neighborhood relationship in high-dimensional space.JSNPP preserves the reconstruction relationship among the neighbors.Compared with the first two algorithms,DL-JSLPP is different in that the deep features extracted from the neural network model are introduced into the traditional joint sparse projection learning model,and the deep learning-based features can improve the performance of the subspace learning algorithm.This paper uses the iterative way to solve the three algorithms'optimal solutions,analyses the algorithms'computational complexity and proves the algorithms'convergence.The proposed algorithms are test on some well-known image daabases.The experimental results show that the performances of the proposed three algorithms are better than some traditional feature extraction methods,based manifold learning feature extraction methods and their new variations.