Research On Some Problems Of Subspace Learning And Its Applications

Subspace learning is a hot topic in the field of machine learning, and has been widely applied in computer science, chemometrics, bioinformatics, etc. When handling data with small sample size but high dimensionality, the commonly used regression, classification models are usually suffering from over-fitting, large parameters estimation errors. However,thought the data are high dimensional, they may distributed on a low dimensional subspace.On this low dimensional subspace, over-fitting and large parameters estimation errors can be avoided. Subspace learning is a well established method to solve these problems. For specific regression, classification tasks, learning the optimal subspace is a key problem in the field of subspace learning.For regression, classification problems, numerous subspace learning methods have been proposed by designing the objective functions and the regularization methods on regression coefficients and projection vector based on various criteria. However, it still remains a challenging topic for us to design the cost function as well as the optimal regularization method for regression coefficients and projection vector in subspace learning to achieve the maximum regression or classification precision.The work of this dissertation concentrates on designing of optimal cost function as well as the regularization method for regression coefficients and projection vector, mainly focusing on three problems, i.e., learning the optimal projection vector in sense of minimum classification error, mean squared error and modeling data with correlations between samples as well as features.The contributions of the dissertation are:1. Research on the problem of extracting the optimal projection vector in linear classification problem, propose a near optimal linear discriminant analysis model. Currently,the linear discriminant analysis models didn't considering whether the projection vector is optimal and dependent on the estimation of mean and covariance matrix from the sample data. In this work, assuming Laplacian distribution, the optimal criterion for extracting the projection vector in the sense of minimum classification error is presented. A linear programming algorithms is proposed to solve the corresponding optimization problem. Instead of estimation to mean value and covariance matrix, the new model relies on the estimation of sample median value and average absolute deviation, which is more robust, especially when the data size is small or outlier exist. Experimental results demonstrate the better classification performance when the data follows Gaussian distribution, Laplacian distribution or suffers from missing attributes.2. Research on the problem of extracting the optimal projection vector in linear regression problem, propose a near optimal partial leas squares model. Assuming the features are noisy, analyze the relationship between mean squared error and projection vector; propose an optimal model for extracting projection vector in the framework of partial least squares; present a near-optimal model and a generalized eigenvalue decomposition methods is proposed to solving the corresponding optimization problem. Experiments on benchmark datasets show that the proposed model can achieve smaller prediction error with less latent variable.3. Research on joint modeling the relationship between samples and the relationship between multiple features, propose a multi-task multi-view learning model in the framework of regression and the corresponding kernel version, explicit solutions of the two models are also presented. The proposed model is applied in the visual tracking problem, simultaneously modeling the similarity between several adjacent frames and the relationship between several different features. Experiments on benchmark datasets show that both tracking precision and real-time problem are significantly improved compared with several state-of-the-art methods.