Sparse Coding On Manifolds

Sparse coding has been used in different applications and received extensive attention. While its excellent performance, there are still two problems:firstly, multiple manifold structure has not been considered well, secondly label information has not been well utilized in classification problems where class labels of training samples are always available. We build three new models to solve these problems.For unsupervised situation, we propose a model called sparse coding on multiple manifolds to solve the problem when different manifolds are very close. We learn an affine space near the data point and treat the intersection of the space and the manifold as the neighborhood of the point. And the absolute values of the coordinates are used to measure the distance between data point and its neighbors. Combining the learned structure with sparse coding model, the structure of the single manifold is preserved, and different manifolds are separated.For supervised situation such as classification, we propose a discriminate sparse coding model with geometrical constraint. Two kinds of graphs, similarity graph and penalty graph, are used to describe the structure of the multiple manifolds. The similarity graph is used to capture the structure of a single manifold. Embedding the similarity graph into sparse coding model, the consistence of the codes along the manifold is enhanced. The penalty graph is used to describe the relationship between different manifolds. Using it as a regularization, the structures denoted by the penalty graph are suppressed, so the codes of different manifolds are separable. The construction of the similarity and the penalty graph is flexible. We implement a specific example using linear representation induced measure. Experiments on some database demonstrate the effectiveness of the model.To resolve the problems that the codes may have different distributions in training data and test data, we bring about the supervised sparse coding with local geometrical constraints. Coupled labeled and unlabeled graphs are used to capture the structure of multiple manifolds. The manifolds denoted by the labeled graph are separable and unlabeled data in the training set can be embedded into the unlabeled graph naturally. Learning a common dictionary, the data in the training set and test set can be embedded into the unlabeled graph using a common method. Besides, the supervised labeled graph enhances the separability of codes. As a result the classification power of the model is improved.