Several Issues In Manifold Learning

In the fields of machine learning and pattern recognition, when we are facedwith tasks like classification or clustering, one major support for our decision is theknowledge of data distribution. Distributions of data in di?erent problems are di-verse. Whether to use, and how to use this information significantly in?uences theperformances of algorithms. Tradition learning methods usually impose restrictiveconstraints on the structure of data distribution, therefore they may be limited whendealing with real world data.In recent years, researchers become interested in manifold based learning to betterexploit the structure of data distribution. There are mainly two motivations for manifoldlearning. One focuses on data's local structure, and the other focuses on the globalstructure. Our research in this article involves both of them. Moreover, to make thealgorithms practical we also proposed some methods for acceleration. Specifically, wehave accomplished the following works in this article.First, we proposed a method named Discriminant Additive Tangent Spaces(DATS) to characterize manifold's local structure. Based on the concept of manifoldtangent spaces, we enhance the traditional linear modeling with a non-parametric for-mulation. And a discriminative training strategy is proposed to promote its perfor-mance in classification tasks. Moreover, we propose a new training algorithm so thatthe model can handle high-dimensional data.Second, based on the graph modeling of manifold, we propose a semi-supervisedapproach to learn the correspondences between di?erent manifolds. We introduce anovel way to supervise and guide the learning process, where the user only has to tellthe algorithm that"A is more like C than B is". Compared to traditional methods,this new type of supervision is more ?exible and easier to obtain, and satisfactoryexperimental results are achieved.Thirdly, we propose a multilevel approach to accelerate the Belief Propagation algorithm on Markov random fields. In this approach, we first recursively reduce thescale of the graph to form a pyramid from bottom up. Then the solution for the top-mostlevel is obtained using belief propagation, which should converge very fast. Finally,the initial solution is propagated from top down until the original problem is solved.Our method not only possesses an intuitively explanation, but also can be justifiedtheoretically.We apply the above three methods respectively to visual object recognition, visualmanifold alignment, and various graph-based manifold learning problems. The experi-mental results show that in most cases they have superior performances over traditionalmethods and algorithms that do not utilize data's manifold structure. In the end, we willconclude our work and discuss future works.