| Recent advances in sensing and storage technology have created many high-volume, high-dimensional data sets in pattern recognition, machine learning, and data mining. The high-dimensional data become more and more complex so that human can hardly understand the inner structure of the data by direct-viewing cognition. People must fall back on machine learning methods to learn from data. Manifold learning method cans effectively learning the inner structure embedded in the high-dimensional data space, it attracts more and more attention in the research area of machine learning and cognitive science.Manifold learning involves the disciplines of differential geometry, statistics, neuroscience and topology, which is a basic, forward-looking research. Since the manifold ways of perception have been proposed by Seung in 2000 from the point of psychological, there are many progresses in manifold learning that have been applied to the field of pattern recognition, computer vision, and image processing. However, because of its complicated mathematical theoretical basis and the cross-integration between disciplines, so there still exist many key theoretical and algorithmic problems. In this dissertation, some key issues on manifold learning have been studied, the main contributions are summarized as follows:1. The dissertation first gave a survey about the manifold learning from five different subareas: neural network, principal manifold, variational methods, mutual information and spectral graph method. And then we introduce the spectral graph methods in detail, which include global methods, local methods, and global alignment of local models. Experimental results show that there are some common issues that determine the effectiveness of the manifold learning method, these issues are listed: How to preserve the local and global structures simultaneously, and how to improve the robustness to sparse data and noise.2. The dissertation proposed a novel manifold learning algorithm called locality and globality preserving embedding (LGPE). LGPE algorithm constructs two relational graphs through manifold distance between data points: A neighborhood graph to describe the local neighborhood information and a non-neighborhood graph to encode the global structure information indicated in the data points far away each other. With the two relational graphs so defined, learning a submanifold can be transformed into solving a constrained optimization problem. LGPE characterizes the intrinsic neighbor relations as well as maximizes the non-local scatter of non-neighborhood data points. Compared with local embedding methods such as LLE, LGPE algorithm considers the global structures of the manifold besides the local structures; while compared with global embedding methods such as ISOMAP, LGPE algorithm can avoid the shortcoming that the intra-neighborhood distances for points in the embedding tend to be inaccurate, and it also incorporates the global information of data in a more flexible way. Experimental results on both synthetic and real datasets show the effectiveness and robustness of LGPE algorithm.3. The dissertation investigated manifold learning algorithms with landmark points and presented a useful variant of LGPE algorithm, called locality and globality preserving embedding with adaptive landmark points (AL-LGPE), based on the analysis and comparison of existed methods. LGPE algorithm can discover low dimensional manifolds from high dimensional data, but it is time-consuming for calculating the shortest-paths distance matrix. To speed up LGPE, the algorithm assigns data point as landmark points according to the distance between it and its neighbors as well as maximizes the sum of distance between landmark points. Point with a large distance has a lager probability to be selected as a landmark point and has a great impact on the embedding performance. The algorithm greatly reduces the probability distribution of landmark points as a line and improves the stability. Experimental results show that the algorithm is effective in practical use.4. The dissertation investigated the framework of graph embedding and extensions and introduced three useful generalizations for LGPE algorithm. LGPE algorithm is only defined on training samples and can only work with vectorized data representations which may not capture well some high-order information in the original data, so it is impractical to argue the superiority of a learning algorithm over all others. For this reason, we introduce three useful generalizations for locality and globality preserving embedding: the linear LGPE, the kernel LGPE, and the Tensor LGPE. As we can see in experiment results, the three variants of LGPE have their own advantages for different circumstances. Hence, the overall LGPE framework is thereby capable of resolving a wide range of problems.5. To narrow down the gap between low-level visual features and high-level semantic concept in face image retrieval system, the dissertation introduce a novel algorithm called the feedback semantic manifold learning algorithm(FSML) by integrating visual features and semantic information to semi-supervised locality and globality preserving embedding algorithm. First,the data of the image database are divided into small clusters by semantic supervised clustering algorithm,so the data of each cluster are similar both in visual features and in semantic information.Then, on the relevant feedback,through users mark the positive and negative samples,the algorithm extends the user's relevance feedbacks in representation of clusters instead of images and aims at maximizing the margin between positive and negative examples at each local neighborhood. After projecting the images into a lower dimensional space, we can obtain a semantic submanifold which can better reflect intrinsic property of image data. Experimental results show that FSML algorithm can effectively improve face image retrieval accuracy. The algorithm can also obtain a better retrieval performance at very low dimensions, which is more suitable to practical application.
|
PDF Full Text Request