Research On Robust Manifold Learning Algorithms

Manifold learning, a typical nonlinear dimensionality reduction technique, is regarded as a hot topic in machine learning, and is widely used in many fields such as face recognition and computer graphics. But its performance depends greatly on the distribution of the original manifold. Moreover, manifold learning methods are very sensitive to noise, and it is also difficult to choose a proper parameter for obtaining ideal results. For dealing with these problems, this paper mainly discusses robust manifold learning methods and their application to visualization and face recognition.First, we briefly introduce the basic concepts for manifold learning, and summarize manifold learning methods into two categories, i.e., the algorithms to preserve global properties and the ones to preserve local properties. Two typical manifold learning methods, ISOMAP and LLE, are elaborately analyzed. Besides, their advantages and disadvantages are also discussed through experiments on artificial data.Second, considering the outliers detection problem, we adopt Box Plot method to detect outliers. Considering the effect of outliers on identifying neighborhood, we propose a new algorithm called Robust Hessian Eigenmaps (RHE, or RHLLE), which is based on Robust Local Principle Component Analysis. This method introduces Reliability that can serve as a reliability-measurement strategy for each neighbor of each point. As a result, the outliers can be detected and its negative influence can be minimized. So the estimation accuracy of the local tangent space coordinates for each point can be consequently enhanced. The experimental results on MNIST handwritten digits show that RHLLE is very robust against outliers.Finally, due to the unsupervised nature of RHLLE, we propose an efficient recognition algorithm called Robust Discriminative Hessian Eigenmaps (RDHLLE) that uses the label information by a new margin maximization criterion. It can not only steadily preserve the data's structure properties, but also make the low-dimensionality data in different classes highly separated. Empirical studies on public face databases, noisy face database and the hybrid of ORL and PIE database thoroughly demonstrate that RDHLLE is superior to popular algorithms for dimension reduction in terms of robustness.