Research On Facial Expression Recognition Algorithm Based On Feature Fusion Of Facial Patch

In recent years,with the development of artificial intelligence technology,users put forward higher requirements on human-computer interaction: the computer should “feel” the emotion of human and make the corresponding feedback.As an important method to recognize emotion,facial expression recognition has become a hot research area.Widely used in human-computer interaction,fatigue driving detection etc..Facial expression recognition has important theoretical value and wide application prospects.This paper is mainly concerened about facial expression recognition algorithm based on facial patch and feature fusion,including data preprocessing,facial patch selection algorithm,facial expression recognition based on Convolutional Neural Network(CNN)and Support Vector Machine(SVM),respectively.In data preprocessing,this paper first introduces the CK+ database,JAFFE database and NVIE database.For the images in databases,Open CV was used to detect face and Face++ API was used to detect the faical landmark points.To improve the contrast of images,histogram equalization was carried out.Rotation correction was also carried out because the obliqueness of face in some images.Finally,spacial normalizatiom was carried out according to the location of left eye.This paper presented an algorithm to select facial patchs based on similarity.The similarity among images is definde vie the average hash distance of them.With this algorithm,this paper selected three kinds of facial patches: left-eye patch,right-eye patch and mouth patch.The size of them are 186 pixels,188 pixels and 96 pixels,respectively.The seleted patches are with small background area,which benefits facial expression recognition.In the process of facial expression recognition based on CNN,for each kind of facial patch,a CNN is trained.The final result is obtained by majority voting of the results of the three CNN,which is a kind of decision level fusion method.The experiments on CK+ database show that the accuracy is improved by 3%-5% using this method.This paper used ten-fold cross validation method to evaluate the perfomace of the preseted method on CK+ database,JAFFE database and NVIE database,and the accuracies are 98.21%,98.41% and 96.51%,respectively.This paper also used SVM for facial expression recognition.Firstly,the Local Binary Pattern(LBP)feature,Histogram of Oriented Gradient(HOG)feature and Gabor feature of each kind of facial patch are extraced.Secondly,the Principal Component Analysis(PCA)and Linear Discriminant Analysis(LDA)are used for dimensionality reduction.After that,the three kinds of features are combined arbitrarily and put together in series connection to train and test the SVM.Similarly,for each kind of facial patch,a SVM is trained.Decision level fusion method is used to get the final result.The exprements shows that LDA performs better than PCA in most cases.However,PCA is more robust.The accuracies on the three databases are 98.29%,93.94% and 87.71%,respectively.By comparison,the accuracy of CNN is higher than that of SVM on complex database.However,the training time of SVM is shorter.