Research On Facial Expression Image And Sequence Recognition

Facial expression recognition is a branch of facial recognition,which refers to the analysis of changes in facial expressions by computers to accurately determine the true emotions of facial expressions.It has certain research value in human-computer interaction,driver fatigue detection,and lies detection.Due to the small difference in expressions of human faces,it is difficult for computers to accurately distinguish various expressions,so it is a very challenging research topic in the field of computer vision.The research of facial expressions can be divided into expression images and expression sequences,both of which can be considered as classification problems similar to pattern recognition,and can be divided into two parts: expression feature extraction and expression classification.In this paper,both emoticon images and expression sequences are studied.Whether it is expression image recognition or expression sequence recognition,it is inseparable from the data set,and in some data sets,the number of various expressions is not balanced,which often leads to problems such as overfitting.In order to solve this problem,this article uses upsampling to make the proportion of samples in various categories tend to balance.Secondly,in order to overcome the problem of the small number of samples in the data set,this article uses online data enhancement to improve the generalization of the model ability.In terms of expression images,the LBP operator is a feature extraction operator with high feature discriminatory power,so it has been widely used.However,the traditional LBP operator extracts the block features for the entire image.Due to the different face poses,the key regions of the faces in different images will not be in the same block after the block.When the size of the data set is small,this problem further exacerbates the overfitting problem caused by the sample's difficulty in covering the feature space;in order to mitigate the impact of this problem,this paper proposes a weighted feature fusion model of face subregions.Some studies have pointed out that when emotional changes occur,the most facial expressions in the entire face are the eyes and mouth areas,and the influence of the remaining areas is much smaller.Therefore,this article cuts out these two areas separately to build a statistical histogram.Feature extraction;in order to make up for the impact caused by the lack of other regions,this paper also uses the entire face as a region to extract the model as a correction;after extracting the features of the sub-regions respectively,this article uses a grid search to determine different sub-regions The weight of the feature fusion,and then perform feature fusion and use the classifier for training.In order to further improve the model performance,this paper introduces the convolutional neural network model and combines the sub-region LBP model for model fusion.In terms of expression sequences,in order to avoid the problems caused by the inconsistency of the number of frames in the sequence,this paper adopts the method of frame averaging and frame expansion to unify the number of frames in the expression sequence.In order to learn the spatiotemporal information in expression changes better,this paper introduces CNN-GRU network and facial motion trajectory network,and uses the method of joint finetuning for model fusion.The CNN-GRU model is built to learn the appearance characteristics of time.Firstly,a convolutional neural network is used to classify each frame to obtain a probability distribution sequence,and then the GRU network is used to build a model for the probability distribution sequence.The 68 key points of the face after normalization are established in order to learn temporal geometric features.The comparison experiments on different data sets prove the effectiveness of the fusion network model proposed in this paper,and the comparison with the methods proposed in recent years also reflects the improvement in recognition rate of this algorithm.