Modeling And Optimization Of Latent Variable Model

As one of the most important fields in machine research, dimensionality reduction has drawn more and more attention and achieved a prodigious progress in the theory research and algorithm. Latent variable model is an efficient tool for dimensionality reduction. This dissertation mainly studies the models and applications of latent variable model in dimensionality reduction. All of the research results can be described as follows:(1) A latent variable model based on local distance preservation is obtained. Latent variable model which establishes smooth kernel mappings from the latent space to the data space cannot keep the points close in the latent space even they are close in data space. To handle this case, a new latent variable model is proposed by utilizing the objective function of the locality preserving projection. Firstly, with the help of the locality preserving projection, a constraint priori of latent variables can be obtained. Then the posteriori probability of the latent variables can be got according to the Bayes theorem. Finally, the positions of the latent variables in the latent space are determined through maximum a posteriori algorithm. Experimental results and comparison with the traditional LVM prove that our proposed method performs well in preserving local structure on common data sets.(2) Two weighted Gaussian process latent variable models (GP-LVM) based on discriminant feature are established. The traditional GP-LVM has two disadvantages, on the one hand, it is an unsupervised method and cannot utilize label information, and on the other hand, the key assumption of the model is that the distributions in all the dimension are identical. In fact, for different tasks, such as classification, the discrimination of each dimension may be different. Take the aforementioned consideration, two weighted GP-LVMs are developed to deal with the inherent drawbacks of GP-LVM. Firstly, a weighted GP-LVM based on discriminant features is obtained by utilizing linear discriminant analysis (LDA) for extracting the weighted values. Then another weighted GP-LVM based on locality preservation can be obtained through extracted weights by local Fisher discriminant analysis.the likelihood in each dimension of the dataset is computed in the GP-LVM. Both of them can utilize the supervised, information and strengthen the discriminant property of the model.(3) The latent variable models for regression and classification are obtained respectively by utilizing the conditional independence in the latent variable models. The former establishes the relationship between the observed space and the output regression space. The latter offers the mapping functions from latent variables to the observed dataset and label set. With the help of dimensionality reduction, the redundancy of the observed data can be removed, and the performances of the regression model and classification model are improved.(4) To further improve the efficient of latent variable models confronted with pairwise constraints, a semi-supervised learning model based on latent variable model is presented. Pairwise constraints are a kind of semi-supervised information and are more easily obtained than label set. Through transferring the pairwise constraints in the observed space to the latent space, the constrained priori information on the latent variables can be obtained. Under this constrained priori, the latent variables are optimized by the maximum a posterior algorithm. The effectiveness of the proposed algorithm is demonstrated with experiments on a variety of data sets. In the experiments, the comparisons with the representative dimensionality reduction models show the advantages of the proposed methods.(5) To deal with the case that the training and testing samples are not indentity independent distribution, a transfer learning framework for latent variable models is proposed based on divergence analysis. With considering the distance of two distributions measured by divergence, we do not need to rebuild the model and only adjust the parameters according to the divergence, which can increase the generalization ability of latent variable models. The proposed model can transfer the knowledge of the training dataset to learn testing dataset. Therefore it can deal with cross-domains tasks. Experimental results on several real data sets demonstrate the advantages of the proposed framework.