Subspace And Manifold Analysis For Data Science

Data analysis has attracted much attention in recent years, with dimensionality reduction, sparsity, and low-rankness proposed to handle data with complex structure.Dimensionality reduction is a method extracting the manifold structure of the data, among which semi-supervised methods are extended from supervised methods, however, these exten-sions usually use all of the unlabeled samples, most of which can not be ensured to play a positive role. Sparsity and low-rankness can be used to analyze the data with the structure of one subspace or mixture of subspaces. Although many priors can induce sparsity, only the ma-trix factorization is often used to enforce low-rankness, leading to the estimated rank dependent on the initial value of the rank. With respect to the data with the structure of multiple subspaces, low-rank representation can ensure the effectiveness theoretically when the subspaces are inde-pendent, however, there is no further analysis about its performance on disjoint subspaces. In addition, linear representation used by the methods of sparsity and low-rankness is difficult to handle the data with the structure of nonlinear manifold.First, the thesis proposes a method extending the supervised dimensionality reduction meth-ods to semi-supervised dimensionality reduction methods, which uses low-rank regression anal-ysis to select the unlabeled data playing a positive role instead of using all. Moreover, combining low-rank regression analysis with spectral analysis can obtain the unsupervised and supervised dimensionality reduction methods capturing the global structure. Second, the paper proposes structure constrained low-rank representation, it can theoretically guarantee its performance not only on independent subspaces but also on disjoint subspaces. In addition, this method can also be applied in graph based semi-supervised learning. Then, we make further research on struc-ture constrained low-rank representation, find the result that the maximal singular value is more appropriate to be combined with sparsity than the nuclear norm and propose dense block and sparse representation. This method minimizing the 2-norm and 1,1-norm of the matrix can per-form as well as structure constrained low-rank representation does without the predefined weight matrix. In the last, the thesis proposes Bayesian low-rank and sparse nonlinear representation. Since its posterior inference by variational Bayes only includes the inner product between the samples, it can be easily kernelized and thus handle the data with the structure of mixing non-linear manifolds by constructing different kernels. Extensive experimental results confirm the effectiveness of our methods.