Matrix Regression Models And Methods With Applications To Robust Image Classification

Regression analysis is a hot topic in machine learning.Characterzing the relationship between data plays an important role in regression analysis.But the practical noise is com-plex,since they can be easily distorted by various factors,such as occlusion and illumination.This results in that traditional methods may cannot sucessfully learn such a relationship.Under the Bayesian framework,this dissertation proposes several novelty robust regression methods by effectively modeling the noise.The major research results are as follows:(1)A nuclear norm based robust image regression method is proposed.Recently,re-gression analysis has become a popular tool for face recognition.Most existing regression methods use the one-dimensional,pixel-based error model,which characterizes the repre-sentation error individually,pixel by pixel,and thus neglects the two-dimensional structure of the error image.It is observed that occlusion and illumination changes generally lead,approximately,to a low-rank error image.In order to make use of this low-rank structural information,this paper presents a two-dimensional image-matrix-based error model,namely,nuclear norm based matrix regression(NMR),for face representation and classifi-cation.NMR uses the minimal nuclear norm of representation error image as a criterion,and the alternating direction method of multipliers(ADMM)to calculate the regression coeffi-cients.A fast ADMM algorithm is further developed to solve the approximate NMR model and it has a quadratic rate of convergence.Some experiments are implemented using four popular face image databases:the Extended Yale B database,the AR database,the Multi-PIE and the FRGC database.Experimental results demonstrate the performance advantage of NMR over the state-of-the-art regression-based methods for face recognition in the pres-ence of occlusionand illumination variations.(2)A robust image regression method is presented.In most current approaches,the error matrix needs to be stretched into a vector and each element is assumed to be inde-pendently corrupted.This ignores the dependence between elements of error.This paper preserves the matrix form of all input images and assumes that the error image caused by partial occlusion or illumination changes is a random matrix variate and follows the extended matrix variate power exponential distribution.This paper reveals the essence of the proposed distribution:it actually alleviates the correlations between pixels in an error matrix E and makes E approximately Gaussian.On basis of this distribution,a Schatten p-norm based matrix regression model with Lq regularization is derived.Alternating Direction Method of Multipliers(ADMM)is applied to solving this model.To get a closed-form solution in each step of the algorithm,two single value function thresholding operators are introduced.In addition,the extended Schatten p-norm is utilized to characterize the distance between test samples and classes in the design of classifier.Extensive experimental results for image recon-struction and classification with structural noise demonstrate that the proposed algorithm works much more robustly than some existing regression based methods.(3)Two nuclear-Li norm joint matrix regression(NLiR)models for face recognition with mixed noise are proposed.The sparse representation based classifier(SRC)has shown a great potential in handling pixel-level sparse noise,while the nuclear norm based matrix regression(NMR)model has been demonstrated to be powerful for dealing with the image-wise structural noise.Both methods,however,might be not very effective for handling the mixed noise:the structural noise plus the sparse noise.By virtue of the attribute of the mixed noise,this paper uses two matrix distributions to fit the mixed noise,respectively.Based on the above two distributions,two robust regression models are proposed.The first model considers the mixed noise as a whole,while the second model assumes that the mixed noise is an additive combination of two independent components:sparse noise and structural noise.The proposed models can be solved by the alternating direction method of multipliers(ADMM).We validate the effectiveness of the proposed models through a series of experi-ments on face reconstruction and recognition.(4)A nesting-structured nuclear norm minimization(NSNM)model is presented.This mehod introduces a nesting-structured nuclear norm and uses it to characterize the matrix variate with structure prior.Meanwhile,a unified framework for solving nesting-structured nuclear norm minimization(NSNM)problem is provided by resorting to an improved sub-gradient method,which integrates an accelerated scheme and a forcing descent step in the traditional sub-gradient method.Compared with F-norm or nuclear norm,nesting-structured nuclear norm can exploit more effectively structure information of a matrix variate since it takes local and global structures of the matrix variate into joint consideration.Moreover,NSNM is applied to matrix regression problem.The extensive experiments for face recog-nition clearly demonstrate the superiority of NSNM over some existing methods.(5)A tree-structured nuclear norm approximation(TSNA)model is presented.The tra-ditional mixed(L1,L2)or(L1,L?)group norm becomes weak in characterizing the internal structure of each group since they cannot alleviate the correlations between variables.Re-cently,nuclear norm has been validated to be useful for depicting a spatially structured ma-trix variable.It considers the global structure of the matrix variable but overlooks the local structure.To combine the advantages of structured sparsity and nuclear norm,this paper assumes that the representation residual with tree-structured prior is a random matrix varia-ble and follows a dependent matrix distribution.The Extended Alternating Direction Method of Multipliers(EADMM)is utilized to solve the proposed model.An efficient bound condi-tion based on the extended restricted isometry constants is provided to show the exact re-covery of the proposed model under the given noisy case.In addition,TSNA is connected with some newest methods such as sparse representation based classifier(SRC),nuclear-L1 norm joint regression(NLiR)and nuclear norm based matrix regression(NMR),which can be regarded as the special cases of TSNA.Experiments with face reconstruction and recog-nition demonstrate the benefits of TSNA over other approaches.(6)The nonparametric Bayesian Correlated Group Regression methods are proposed.The existing sparse Bayesian models generally use independent Gaussian distribution as the prior knowledge of the noise.This contradicts to some practical observations in which the noise is long-tail and pixels of the noise are spatially correlated.To adapt more practical noise,this paper proposes to partition the noise image into several matrix groups and adopt a long-tail distribution,i.e.,the scale mixture of matrix Gaussian distribution,to model each group to ex-ploit the intra-group correlation of the noise.Under the nonparametric Bayesian estimation,the low rank induced prior and the matrix Gamma distribution prior are imposed on the covariance matrix of each group,respectively.This induces two Bayesian Correlated Group Regression(BCGR)methods.What is more,the proposed method is extended to the unknown group struc-ture.BCGR provides an effective way to automatically fit the noise distribution.In particular,it integrates the long-tail attribute and structure information of the practical noise into modeling.Therefore,the estimated coefficients are better for reconstructing the desired data.The proposed methods are compared with some newest methods,which can be considered as the special cases of BCGR.BCGR is applied on the image classification and utilize the learned covariance ma-trices to construct a grouped Mahalanobis distance to measure the reconstruction residual of each class in the design of classifier.Experimental results demonstrate the effectiveness of BCGR.