Algorithms For Structure-enforced Matrix Factorization With Applications

Matrix factorization has always been one of the hot research fields not only as a fundamental mathematical concept but as a data analysis tool.For decades now,matrix factorization has become a basic data processing method with applications in such fields as computer vision,doc-ument clustering,audio signal processing and recommender systems.Recently,many different matrix decompositions with specific structures have arisen with good performance on data pro-cessing in practice.In this paper,we mainly focus on one kind of matrix factorization called structure-enforced matrix factorization,design a uniform algorithm framework and study on its applications to many problems.In Chapter 1,we briefly summarize varieties,evolution and recent work on matrix factor-ization and present a uniform model on structure-enforced matrix factorization(SeMF).In Chapter 2,we propose an alternating direction and projection algorithm for SeMF in terms of the framework on existing matrix factorizations and the popular alternating direction method of multipliers(ADMM).We discuss the issue of projections onto several popular struc-ture sets that need to be performed in the algorithm for solving relevant problems.As for the critical issue on choices of penalty parameters,we propose a strategy for adaptively updating penalty parameters.This dynamic scheme enables the resulting algorithm to work quite well in our extensive numerical experiments,in terms of both reliability and efficiency.In Chapter 3,we apply the SeMF algorithm to dictionary learning for sparse representation and compare it with the well-established algorithm K-SVD.Numerical results show that SeMF algorithm tends to perform better than K-SVD on the required number of samples,computing time and robustness to the noise level.Moreover,we present a deep SeMF model for deep dictionary learning and devise an ADMM-extended algorithm.Numerical test for a deep semi-NMF problem shows the efficiency of our algorithm.In Chapter 4,we apply the SeMF model to feature extraction on ORL face datasets and a swimmer datasets.Fistly,we impose different sparsity level on structure constraints,test SeMF on ORL database and obtain corresponding part-based extraction.Then,we design four different structure sets according to properties of the swimmer data and factor the data using our SeMF algorithm,respectively.Results show various torso and limbs extractions.In Chapter 5,since a number of widely used clustering and classification methods can be ac-commodated into nonnegative matrix factorization framework with some variations,we present an extended SeMF model to clustering and classification problems.Through experiments on M-NIST datasets and Brodatz dataset,we conclude the new model can produce favorable clustering results and demonstrate its versatility.