A Study Of Fisher Discriminant Analysis Based On L₁ Norm

Pattern recognition aims to use a computer to simulate or to implement human learning and identification based on the characteristics or attributes of the study object.Feature extraction plays an important role in pattern recognition,which can effectively alleviate the problem of “curse of dimensionality” that often appeared in the filed of pattern recognition.Fisher linear discriminant analysis?FLDA?as a classical method of feature extraction has been widely applied in many aspects such as biometrics,image retrieval and text classification,etc.By maximizing of the Fisher criterion,FLDA seeks a subspace to achive better separability,that is the inter-class scatter of the data in the subspace is the largest,and the intra-class scatter is the smallest.However,the distance in the Fisher criterion is a measure of the L₂ norm metric,which is sensitive to outliers and is lack of robustness.In recent years,the discriminant analysis based on robust distance such as L₁ norm has attracted some researchers' attention.Fisher linear discriminant analysis algorithm based on L₁norm?FLDA_L1?,which not only improves the robustness of the FLDA,but also avoids the rank limitation.However,solving FLDA_L1 is challenging due to its nonconvexity.Based on the framework of concave-convex procedure?CCCP?,on one hand,we propose a new algorithm of FLDA_L1 and verify that FLDA with L₁ norm is more robust than L₂ norm in feature extraction with some examples.On the other hand,we extend FLDA_L1 to nonlinear kernel case,so obtain a new model and algorithm for the kernel Fisher discriminant analysis based on L₁ norm?KFDA_L1?.The expriments in visualization and classification accuracy show that KFDA_L1 is better than KFDA and FLDA after feature extraction on some artificial datas,some UCI datas and some image datas.Overall,it is better for Fisher discriminant analysis to use L₁ norm instead of L₂ norm to overcome the problem of rank limitation and improve robustness in feature extraction.