Linear Discriminant Analysis Based On Clustering Regularization

In recent years, multimedia and netwok techniques develop rapidly, and itpromotes the number of image data to increase at an amazing rate. So how to obtaionthe useful information quickly and accurately from a lot of image data becomes aurgent problem, and dimension redution technique which is as a kind of solution hasbeen a very hot research topic. So far, there are two important methods, one isPrincipal Component Analysis (PCA) and the other is Linear Discriminant Analysis(LDA).LDA is as a supervised dimensionality reduction technique, its main idea is thatfinding an optimal projection direction firstly, and that then projecting the sample datato this direction to ensure that the new between-class dispersion is largest and the newwithin-class dispersion is smallest respectively and simultaneously. However, whenthe number of training samples per class is small, LDA has serious overfittingproblem. The main reason is that the between-class and within-class scatter matricescomputed from the limited number of training samples deviate from the underlyingones greatly.To solve the above problem without increasing the number of training samples,we propose making use of the structure of the given training data, and using k-meansalgorithm to generate the new clustered data, then calculating the between-class andwithin-class scatter matrices of the new clustered data, and using them to regularizethe original between-class and wihin matrices, respectively and simultaneously. Thecontributions are inversely proportional to the number of training samples per class.The advantages of the proposed method become more remarkable as the number oftraining samples per class decreases.Experimental results on AR face databases, FERET face database, andCarreira-Perpinan ear database demonstrate the effectiveness of the proposed method.