Research On Video Expression Recognition Based On Fusion Algorithm

This paper explores the following aspects of video-based facial expression recognition,including:(1)a new feature descriptor,and fusion of multiple features for facial expression recognition in video;(2)A method based on LSTM for processing facial expressions in video,and based on which two LSTM models of appearance features and geometric features are combined to recognize facial expressions.In the first study,a new feature and multi-core learning application was proposed to combine facial features in video.A new feature descriptor,called "Directed Gradient Histogram from Three Orthogonal Planes"(HOG-3D),is proposed to characterize facial appearance changes.A new efficient geometric feature is also proposed to capture facial contour changes,and the role of audio methods in emotion recognition is discussed.Multi-feature fusion can be used to optimally combine different features.The experimental results show that compared with other methods in recent years,this method is very effective in dealing with facial expression recognition problems of video in laboratory controlled environment and outdoor environment.The second study is a fusion model of two separate face analysis systems,both of which use long and short memory(LSTM)networks,a video-based face verification model(3L Model)and a spontaneous facial expression recognition model.(CL Model);develops an LSTM-based model to extract temporal and spatial features of facial expressions,derived from the appearance and geometric features of video expression data,respectively.On this basis,two LSTM facial expression recognition models are used for weighted fusion.The output of LSTM1 is based on the appearance characteristics of the input image sequence,while the output of LSTM2 is based on the geometric features of the input image sequence.The method performed an experiment of expression detection in facial expression databases CK+,JAFFE,FER2013,MMI,and BP4 D.The experimental results are compared with the previousresearch data,which proves the effectiveness of the proposed new method.