Study Of Non-greedy Algorithm Based Discirminant Analysis

With the development of information technology and increase in the amount of data,noises and data redundancy are becoming more and more.It has always been a topical issue to extract the features effectively from the noisy data in the fields of pattern recognition,machine learning and data analysis.Linear discriminant analysis is a classical feature extraction method and is widely used in the area of feature extraction.Since most of the existing feature extraction algorithms based on linear discriminant analysis are using L2-norm square to measure,which overemphasizes on outliers,the robustness thereof is not good.This paper studied two robust discriminant feature extraction algorithms based on L1-norm and L21-norm.The main contents of this paper are as follows:1.The existing L1-2DLDA uses the greedy algorithm to solve the projection vectors,so it does not optimize the objective function to the maximum,and the projection direction therein is not relevant and the optimization time thereof is long.To address these problems,this paper studies a non-greedy iterative algorithm to solve the Tr-L1-2DLDA problem by constructing an auxiliary function,in combination with sub-gradient method and Armijo search method,to optimize the projection matrix as a whole.By applying the algorithm to the PIE,Extended Yale B and AR database for classification,it can be concluded that algorithm presented in this paper can extract more accurate features,is more robust to noises,can obtain larger objective function value with shorter running time and higher recognition rate,and it is locally convergent.2.The existing LDA-L1 uses the greedy algorithm to solve the projection vectors.In addition,L1-norm is not rotation invariant,cannot characterize the discriminant geometric structure of data and is complicated to solve.To address these problems,this paper proposes L21-MMC and further studies a non-greedy iterative algorithm to solve the L21-MMC problem with the derivative of L21-norm,in combination with sub-gradient method and Armijo search method to optimize the projection matrix as a whole.By applying the algorithm to the PIE,Extended Yale B and AR database for classification,it can be concluded that algorithm presented in this paper can extract more accurate features,is more robust to noises,can obtain higher recognition rate and is locally convergent.