Feature Extraction Using Correntropy Based Average Neighborhood Margin Maximization

Feature extraction is the core research issue in the fields of pattern recognition andmachine learning. Its aim is to overcome the curse of dimensionality and overfitting problems.Average Neighborhood Margin Maximization (ANMM) is a recently proposed featureextraction method. It can effectively deal with the small sample problem faced by LinearDiscriminant Analysis (LDA). ANMM can pull the homogeneous data points as near aspossible and contemporarily disperse the inhomogeneous data points.In this dissertation, two main research studies have been conducted to improve theanti-noise and generalization abilities of ANMM.1. Correntropy based ANMM (CANMM) is proposed basing on ANMM. CANMMutilizes correntropy to substitute the Euclidean distance in the traditional ANMM,which makes CANMM more robust against noise. To enhance the solving efficiency,half-quadratic optimization technique is applied to solve the optimization problem ofCANMM.2. Two types of regularized correntropy based ANMM (RCANMM) are proposed.Regularization method generates the restriction effect on itself by utilizing the normof its constraint parameters. Moreover, regularization method can improve thegeneralization ability of CANMM. Among the two presented approaches, L2-normbased regularization term is helpful to enhance the anti-overfitting ability ofCANMM and reduce the appearance of overfitting in some degree. In addition,L1-norm based regularization term is introduced and the method of surrogatefunction is used to solve the corresponding optimization problem. At the same time,the Bayesian method is utilized to implement the automatic selection of theregularization parameter.Experimental results on the three benchmark face databases, i.e., AR, YALE-B and ORL,demonstrated that the proposed methods can improve the generalization and anti-noiseabilities of ANMM.