Research On Facial Expression Recognition Based On Feature Depth Fusion

Facial expression recognition is an important research topic in computer vision.With the development of artificial intelligence,it has broad application prospects in the fields of human-computer interaction,online education and virtual reality,and so on.Aiming at the low recognition rate of existing facial expression recognition methods,which are easily affected by external environmental factors,such as light,noise and others,this thesis proposes a facial expression recognition method based on feature depth fusion.This method extracts multi-features from the pre-processed facial expression image,and then feeds them into the improved auto-encoder network to be training.Finally,the Softmax classifier is used to classify and recognize the expressions.The main work are as follows:A preprocessing method based on irregular segmentation of facial landmarks is proposed.The facial expression image is segmented into 15 irregular polygon feature blocks by locating and connecting the landmarks,which conforms to the physiological structure of the face and is helpful to extract the facial local information.A feature extraction method named Pixel Difference Local Directional Number Pattern(PD-LDN)is proposed.PD-LDN uses symmetrical Robinson mask to calculate the edge response values of local neighborhoods.According to the calculation results,two main directions are selected for encoding,and the pixel values in the encoding direction are binaries by the adaptive threshold.PD-LDN can extract detailed local texture information and make the characterization of features powerful.A feature depth fusion method based on improved auto-encoder network is proposed.The proposed PD-LDN algorithm,the seventh-order moment and the bag of words model are used to extract the texture,geometric and semantic features of facial expression images respectively.The information of facial expression images is described from many aspects.Then the three features are feed into the improved autoencoder network for depth fusion,and the loss function is used to evaluate the fusion effect of the network,so that the network can achieve good result for feature fusion.Finally,the Softmax classifier is used to classify and recognize facial expressions.The proposed method is compared with the current mainstream methods in JAFFE and CK + databases.The experimental results show that the proposed method can effectively recognize facial expressions in the presence of multiple interference factors,and has strong robustness.