Research On Facial Expression Recognition Based On Multi-features

Facial expression recognition is an important technology in computer vision and has become very popular in recent years.Facial expression recognition technology has been successful in different fields,but it is still a challenging task because of the different emotional expressions of each person,skin color,brightness,posture and background.When solving the problem of facial expression recognition,the feature extraction ability of a single feature descriptor is limited,and the general method extracts the expression feature from the entire face,which may ignore the local area with rich emotions.Therefore,this thesis has done related works on the above issues.The following is the main work of this thesis.(1)This thesis mainly introduces the background and significance of the study and expounds the study status of facial expression recognition.Then the general process of facial expression recognition is introduced in detail.The face detection and alignment methods are introduced briefly.The commonly used image preprocessing methods are expounded and several basic facial expression feature extraction methods are introduced.Finally,the feature classification methods are introduced.(2)This thesis proposes a facial expression recognition method that iteratively fuses classifiers based on multi-orientation gradient calculated HOG(MO-HOG)features and deep-learned features.Diagonal orientation gradient calculated HOG(D-HOG)is a complementary part to histogram of oriented gradient(HOG),which is proposed to obtain the diagonal gradient information and combines HOG to form a novel feature descriptor MO-HOG.Our method extracts MO-HOG features from whole facial images and expression rich local facial images.Meanwhile,Deep-learned features are not reliable enough on small databases but contain high-level semantic information,so the deep network is designed to extract effective deep-learned features.In addition,a classifier fusion method based on an optimization algorithm is proposed,and the best fused classifier is obtained through iteration.The experiments are evaluated on the public databases(CK+ and JAFFE).The proposed method shows the effectiveness in facial expression recognition and outperforms the state-of-the-art methods.The recognition accuracy is 97.70% on the CK+ database and 97.64% on the JAFFE database.(3)Emotionally rich local facial expression features intuitively contribute to the extraction of facial expression features.This thesis proposes a facial expression recognition method based on facial part attention mechanism.In this thesis,the attention mechanism is used to extract the emotionally rich local area features,which is complementary to the overall facial features for better facial expression recognition.Firstly,this thesis proposes a cluster-based facial landmarks selection method to select facial landmarks that have commonality and best reflect local regional emotions.Then,this thesis designs a facial part attention mechanism Convolutional Neural Network FPA-CNN.The FPA-CNN is composed of two parts.The first part is the object network for extracting the whole facial features,the second part is the part attention network,and the attention mechanism is used to extract the local part features.Finally,the two classifiers are trained and the classifiers are fused.The method of this thesis was carried out on the RAF facial expression dataset,and achieved 87.26% recognition accuracy,which proved the effectiveness of the method.