Research On Facial Expression Recognition Based On Video Sequence

As a form of non-verbal communication,facial expression is an important way of expressing human emotions.Studying human facial expressions deeply is of great significance to understand human inner psychological and emotional states.With the continuous development of science and technology,the application of facial expression recognition technology for video sequences has become more and more widespread.Since facial expression is a kind of facial appearance change caused by human emotions,it contains a process of change.Due to the static facial image contains limited expression information,the dynamic video sequence can better contain rich expression context information and is more consistent with the mechanism of human facial expressions.The mechanism by which facial expression is produced,and thus its description is more true and accurate.So the expression recognition of video sequences has more research significance.This paper focuses on the facial expression recognition of video sequences,starting with how to extract the spatio-temporal domain information of video sequences,this paper proposes several innovative methods and verifies the validity of the proposed scheme in the domestic and foreign facial expression databases.The main work of this paper are as follows:(1)The essence of facial expression is the movement process of the facial area caused by human emotions,so the dynamic features are more suitable for describing facial expressions.For the video sequence expression recognition,the static feature descriptor can not effectively reflect the change process of facial area movement.This paper proposes a facial expression recognition method,which combining dynamic texture information and motion information,based on the static description of Weber Local Descriptor(WLD),learning from the principle of local binary pattern on three orthogonal planes(LBP-TOP),introducing the time dimension,a spatial-temporal Weber Local Descriptor(STWLD)that reflects spatial-temporal domain information is proposed.Moreover,using block-based optical flow histogram features(BHOF),The feature describes the motion information of the face regions between adjacent frames.Finally,the facial expressions of the fused dynamic texture information and motion information are classified using a support vector machine.The experimental results show that this method achieves better results than a single feature description.(2)For the manual feature extraction method,the two phases of feature extraction and classification are studied and calculated separately.Each stage is independent of each other,which is not conducive to the optimization of the algorithm and the improvement of recognition performance.This paper adopts deep learning method to recognize facial expressions and proposes a video expression recognition method based on parallel convolutional neural network.Since the original CNN does not consider the time dimension information and introduces the time dimension,a 3D convolutional neural network structure is designed to extract the local time domain information of the video sequence.In addition,to make up for the insufficiency of the network to extract the global time domain information,the CNN-RNN network is added,according to the iterative nature of RNN uses the information of the previous frame of the video sequence to influence the subsequent frames and effectively extracts the time-related information,thus describing the global time domain information of the video.The two neural networks fuse the local and global temporal information of the video sequence,which effectively enhances the ability to describe video sequence information.The experimental results show that this method is superior to the method in the single network and related literature,and effectively enhances the recognition ability.