Intrinsic Structure Preserving Non-negative Matrix Factorization Method For High Dimensional Data

Non-negative matrix factorization(NMF) is a hot research topic in the field of pattern recognition and machine learning. Unlike the other matrix decomposition methods. NMF factorizes a nonnegative matrix into the product of two nonnegative matrices(one is basis matrix, another is weight matrix). The nonnegative constraints lead to a sparsely distributed data coding that might be useful for extracting parts-based representation of data patterns with low feature dimensionality. In this regard, NMF agrees with human visual perceive system. The classical matrix decomposition, such as Single Value Decomposition(SVD) is to decompose a matrix into the product of three matrices, but the matrices contain negative elements. As the result, there is no physical meaning of the decomposition. Thus, NMF method has been widely used in image processing, data mining, environmental monitoring, chemo-metrics and multimedia analysis. From the perspective view of dimensionality reduction, we may choose a proper parameter such that can obtain a low-dimensional representation by the decomposition. In this work, we focus on NMF model and algorithm. We propose an intrinsic structure preserving non-negative matrix factorization model and algorithm for dealing with high dimensional data.In this paper, we add a regularization term to the objective function of the standard non negative matrix factorization model, and a new non-negative matrix factorization model is proposed to maintain the intrinsic structure of high-dimensional data in the low-dimensional space. The regularization term attempts to describe the mismatch between the distribution of the original data and the low-dimensional representation. The Gaussian and Cauchy kernel density estimation is used to estimate the distribution of the original data and the low-dimensional representation, respectively. The Kullback Leibler divergence(K-L divergence) is used to measure the mismatch between the distribution of the original data and the low-dimensional representation. In order to preserve the intrinsic structure of the original data, the regularization term needs to be minimized. A gradient descent algorithm is proposed to solve the regularized NMF model. The proposed algorithm is to calculate the optimal step size in each iteration such that the objective function is the largest descent.