Research On Dynamic Facial Expression Recognition Algorithm Based On Image Sequences

Facial expression recognition is an important research topic in the field of computer vision.Most of the research focuses on the static facial expression recognition task with static expression image as the research object.However,facial expression is a dynamic change process,a single static facial expression image cannot fully capture human emotional variation.In contrast,facial expression video or facial expression image sequences can convey human emotional change more completely because it contains rich texture information and motion information related to facial expression variation.Therefore,the research of dynamic expression recognition based on image sequence has great research value.The main work of this dissertation is as follows:1.The regional feature extraction in facial expression image is studied deeply.As different facial expressions have different structure and texture information in different regions of the face.Different convolution kernels should be used to process different regions when using convolution kernels to extract facial features.Specifically,we design a multi-scale region feature learning module and validate it on the facial expression data set which collected in natural scenes.The addition of multi-scale region learning module indeed improve the model's ability to represent facial expression features.2.A dynamic facial expression recognition algorithm based on heterogeneous network fusion is proposed in this paper.It solves the problem of bad performance caused by redundancy and noise interference of features extracted directly from expression sequences by 3D convolution neural network.The algorithm integrates two kinds of network models: 3D spatiotemporal network and static network.The former uses facial expression image sequences as inputs directly to extract coherent spatiotemporal information between adjacent frames,while,the latter uses key frames of facial expression sequences as inputs to extract static features,and then uses model fusion strategy to make up for the deficiency of the former in extracting sequence features and improve the performance of dynamic facial expression recognition.Finally,the experimental results on CK + datasets and Oulu-CASIA datasets show that our proposal has excellent recognition results.3.The problems and shortcomings of the proposed dynamic facial expression recognition algorithm based on heterogeneous network fusion applied in real scenes are analyzed in depth.Then,a dynamic facial expression recognition algorithm based on temporal relation reasoning is proposed.The algorithm samples sparsely the expression sequences of different lengths,and then,we design a multi-scale regional feature extraction network to extract semantic features,and then uses temporal relation reasoning module to model sparse expression sequence's temporal context information innovatively,so as to get the category of expression sequence.Finally,the experimental results show that the algorithm has a good recognition rate for dynamic expression recognition in natural scenes,and can achieve real-time performance.