Research On Video Facial Expression Recognition Algorithm Based On Deep Learning

Video facial expression recognition is an important branch of emotion computing.By analyzing facial videos to predict human dynamic expressions,we can obtain clues about human emotional states,mental activities and so on.Video facial expression recognition plays an important role in promoting user-centered human-computer interaction and realizing intelligent emotion analysis and application.Based on the method of deep learning,this thesis focuses on the task of video facial expression recognition.By jointly mining the information of space and time dimensions,feasible models and algorithms are constructed to improve the accuracy of expression recognition.The research results are mainly reflected in the following aspects:1.From the perspective of spatial features,the thesis proposes a video facial expression recognition algorithm based on attention mechanism.The expression change is closely related to the specific sensitive area of the face,and deep network structure will introduce a large amount of redundant information.Considering the above factors,the algorithm uses spatial attention unit to divide the importance of feature space domain.While strengthening the expression of key local features,it suppresses the output of redundant spatial features and.improves the interpretability of expression features.At the same time,to solve the problems of face pose and model complexity,we use a deep network architecture guided by facial landmarks to reduce the impact of identity information and improve the discrimination of expression features.In addition,aiming at the problem of over fitting in small datasets,the transfer learning method is used to transfer the ability of network feature extraction to the task of facial expression recognition.We conduct relevant experiments on CK+and AFEW datasets,and visualize the attention distribution map,showing the attention to different areas of the face.It can be seen that the algorithm considers the changing process of facial expressions.It processes variable-length expression videos to generate fixed-dimension video feature representations,and thus completes the task of video facial expression recognition.2.Based on the time information,a video facial expression recognition algorithm with enhanced temporal features is proposed.Expressions are usually presented only for a period of time.However,most algorithms focus on extracting feature representation from the whole video,lacking research on key temporal information.Therefore,in order to reduce the spatiotemporal redundancy of image frames and the dependence on manually annotated peak frames,the algorithm designs a loss function based on the multi classification cross entropy function.The key frame subset with the richest expression information is obtained through the optimization process of the loss function.In addition,in order to enhance the correlation of temporal features,the Bi-directional Recurrent Neural Network is used to fuse the temporal and spatial information of key frames.The effectiveness of the algorithm is verified by experiments.And the recognition accuracy on the CK+and AFEW datasets reached 98.47%and 52.48%,respectively.Experiments show that the algorithm can adaptively learn the key frames that contribute the most to the performance of expression recognition,and has obvious advantages in strengthening the expression of temporal information.In summary,the algorithms constructed in the thesis can extract and fuse the features with strong expression discrimination from both spatial and temporal dimensions.They have good classification ability in the task of video facial expression recognition and open up new ideas for the research of extracting heuristic spatio-temporal information in the future.