Nonnegative Matrix Factorization Algorithm Based On The Regularized Method And Its Applications

In recent years, with the wide applications of recognition technology in natural resource analysis, physiological changes, weather forecast, navigation, map and terrain matching, environmental monitoring and so on, many theories and methods have been applied to recognition. Nonnegative matrix factorization is one of the representative method of them has been widely concerned. From the point of view of pattern recognition, nonnegative matrix factorization is essentially a subspace analysis method, and its essence is a method for feature extraction and selection. The main idea for nonnegative matrix factorization is that:a suitable subspace (feature subspace) in the sample space is found, and the high dimensional sample is projected to low dimension subspace, and the essential features of the sample in the subspace are obtained, and then classification is realized by these features. As a data processing technique, the method of nonnegative matrix factorization has revealed the essence of describing data, and has been widely applied to the researches on face detection and recognition, text analysis and clustering, and so on. In recent years, manifold learning becomes a hot study topic in the field of machine learning and pattern recognition. It aims is to seek a low-dimensional smoothly manifold in a high-dimensional observation data space. Manifold regularized matrix factorization methods are more widely used to extract feature. In this paper, in order to study the method of manifold regularized matrix factorization, the dissertation tries to learn effective data representations by exploring the underlying data structure such as the geometrical structure, and tries to utilize a small amount of available supervised information such as scarcely labeled data. Also in order to enhance the sparseness of the factor matrices, the dissertation tries to improve the NMF algorithm.The major research work in the dissertation includes the following several aspects:(1) The dissertation explores the framework processes of the ANLS-NMF (Alternating Nonnegative Least Squares for NMF) algorithm, the method of projected gradient with Armijo Rule and the algorithm of the nonnegative matrix underapproximation. In order to reduce the time complexity, a new algorithm for nonnegative matrix factorization, denoted projected gradient nonnegative matrix underapproximation (PGNMU) is proposed. The experiments on different database validate the effectiveness of the proposed algorithm, and show that it can learn better parts-based representations of data than the competitions.(2) As the smoothed LO norm of the factor matrices can reflect the sparseness intuitively and it is easy to be optimized, we consider NMF on orthogonal subspace with smoothed LO norm constraints, called NMFOS-SLO, and its application to the task of clustering, where an orthogonality constraint and the smoothed LO norm constraints are imposed on the nonnegative decomposition of an inputting data matrix. We develop new multiplicative updates for NMFOS-SLO. Experiments on different datasets show our method perform better in the task of clustering, and sparseness of the factor matrices, compared to the other method.(3) Considering the fact that the outlier samples in the patterns may have some adverse influences on the classification result, and based on the method framework of NMF-KNN, we develop a novel NMF algorithm regularized by intra-class and inter-class fuzzy K nearest neighbor graphs by using fuzzy sets, leading to NMF-FKNN. By introducing two novel fuzzy K nearest neighbor graphs, NMF-FKNN can contract the intra-class neighborhoods and expand the inter-class neighborhoods in the decomposition. This method not only exploits the discriminative information, but also uses the geometric structure in the data effectively. Also we develop simple multiplicative updates for NMF-FKNN and present monotonic convergence results.(4) A graph dual regularized concept factorization framework is proposed, and can make use of the geometrical and low-rank structures of the given data. The framework can simultaneously consider geometrical structures of both the data manifold and the feature manifold, whose information is encoded by constructing two nearest neighbor graphs, respectively. Under this framework, a graph dual regularized concept factorization (GCF) model is presented, and its iterative multiplicative updating scheme is also provided. Finally, the convergence proof of the proposed iterative multiplicative updating schemes is provided.