Locality Preserving Manifold Learning And It's Application To Visual Information Analysis

Vision is an important part of human intelligence, while visual information is the most complex and useful data in human perception system. It is one of the most significant challenges in fundamental and applied research endeavor to let computers have the ability of recognition like human vision system. Since computer vision theory and techniques is far less perfect, the research on visual information analysis is still an interesting topic.Although the great amount high dimension, unstructured visual data can provide more detailed information, the curses of dimensionality caused by dimension inflation is inevitable. Dimension reduction turns to be effective, even desirable. The traditional linear dimension reduction methods, including the Principal Component Analysis (PCA), multi-dimensional scaling (MDS) and linear discriminant analysis (LDA), etc, are to design the feature vectors using linear model on high dimensional space. Their advantages are simple operation, simple transformation function, and effective to linear structure. But the high dimensional data in practical (such as visual information) is mostly nonlinear. It is hard to explore the structure and the correlation or to reveal the manifold distribution of the high dimensional data by linear method.Locality preserving manifold learning methods, which have been widely studyed and applied to nonlinear dimension reduction of high dimensional data, handwritten character recognition, and face and pose recognition, have been demonstrated with favorable performance. However, due to the complexity of high dimensional visual information, these methods inevitably exists some problems. For example, how to embed temporal information to manifold learning, and multi-class image sets, or multi-shot video analysis, etc. This paper attempts to study the relevant manifolds algorithm in depth. The maniold learning algorithms to visual information are proposed. The main work and contributions are as follows:(a) Time-embedding methods for manifold learningSome visual information, such as video, the temporal information is an important clue for semantic analysis. But manifold learning is independent of temporal information. So how to embed time information to manifold learning is worthy of study. Firstly, time-embedding on two dimensional locality preserving projection (TE-2DLPP) is proposed. A summary storyboard of a video is created in TE-2DLPP feature subspace, and the experimental results are encouraging. Secondly, a new concept named video manifold feature is proposed. The proposed manifold feature extraction method is applied to shot transition and video trajectory analysis, which provides a new tool for video analysis.(b) Multi-class manifold learning method based on locally linear embeddingLocally Linear Embedding (LLE) requires that the input high dimensional data must lie on a smooth and well-sampled single manifold. Therefore, it will fail to find the embedding on multi-manifold (multi-class data) which always appears in the practical data such as multi-class image set and multi-shot video. A new multi-manifold learning method based on LLE is proposed. The experimental results demonstrate that the proposed method can achieve good embedding result for multi-class image set and multi-shot video.The LLE-based methods can't be directly applied to the new test data because they do not adopt the dominant mapping. Meanwhile, they have limitations for classification. A classification method for multi-class manifold is proposed, and good performance is achieved.(c) Locally linear embedding based semi-supervised manifold learningIn many applications of practical machine learning and data mining, only a few labeled training samples are usually available although a large number of unlabeled training samples are in hand. To solve the problem, a novel semi-supervised manifold learning method based on locally linear embedding is proposed to automatically exploit unlabeled samples to help supervised learning. The basic idea is not only to preserve intra-class data neighbouring relationship in the processing of dimension reduction but also to predict the label of a data point according to its neighbors. Different from existing approaches, our approach provides a novel graph structure construction method by modified minimum spanning trees instead of k-NN graph. We present dual weights:reconstructing weights for finding the embedding, and derivative weights for class label propagate. The experimental results on synthetic data S-curve, multi-class data sets and transductive classification demonstrate the effectiveness of the proposed approach.(d) Unifying framework for Locality Preserving Manifold LearningThe locality preserving manifold learning mainly depicts local geometrical properties through establishing the local model, then integrates and aligns all the overlaps of the local models to find the inherent global geometry, and abtain the global low-dimensional coordinates finally. After analyzing the typical algorithms, such as LLE, LTSA and LE, a unified locality preserving manifold learning framework is proposed. At last we prove the relations between these typical algorithms and the framework.