| As an important component of emotion recognition,facial expression recognition has been widely used in various scenarios such as human-computer interaction,medical diagnosis,security monitoring,and educational training.With the continuous development of deep learning technology in image processing,deep learning-based facial expression recognition algorithms have become a research hotspot.Among them,Convolutional Neural Networks(CNN)have shown great potential in facial expression recognition task.Existing algorithms have reached high recognition accuracy in controlled laboratory environment.However,in natural environment,facial expression recognition still faces challenges due to subjective and objective factors.On the one hand,the classical CNN models have difficulty accurately representing facial images with occlusion or pose variation in natural environment.On the other hand,the commonly used cross-entropy loss is insufficient to constrain the network to learn discriminative expression feature from subtle expression.To improve the accuracy of facial expression recognition in natural environment,aiming at the above problems,this paper optimizes the network structure and loss function based on the Residual Network(Res Net).The goal is to enable the facial expression recognition model to extract robust facial feature and effectively discriminate subtle expression.The main contributions of this paper are as follows:(1)In order to accurately represent facial images in natural environment with occlusion or pose variation,an attention-modulated contextual spatial information network(ACSI-Net)is proposed,in which the contextual convolution is adopted in Res Net to explore and integrate global contextual spatial information.Additionally,the coordinated attention module is embedded to guide the network in obtaining local salient feature.Experiments on facial expression datasets in natural environment verify that the proposed model can extract robust facial feature from both global and local levels,and the recognition accuracy is improved by1.06% ～1.31% compared with the baseline method.(2)In order to learn discriminative facial features from subtle expressions,a new joint loss function design method was proposed to improve the cross entropy loss function of ACSINet model.Firstly,the weights of different polynomial bases in the Taylor expansion of cross entropy loss function are adjusted to ensure the learning strength of the model for categories with less samples.Secondly,using the metric learning theory,sparse center loss based on Euclidean distance is designed,which is combined with the improved cross entropy loss to supervise the training of the model,so as to increase the feature differences between classes and at the same time,the same features are aggregated.The experimental results show that the proposed joint loss function can constrain the network to effectively discriminate expression.Under the constraint of joint loss,the recognition performance of ACSI-Net model is improved,and the recognition accuracy of RAF＿DB and Affect Net datasets is 87.87% and 65.27%,respectively.(3)In order to further optimize the expression feature space,an adaptive feature enhancement-based facial expression recognition model is constructed.An adaptive feature modulator(AFM)is added to the proposed ACSI-Net to achieve adaptive modulation of feature through feature grouping,refinement,and aggregation.The AFM reasonably utilizes the synergy between channel attention and spatial attention and provides more effective guidance for the sparse center loss.Compared with the existing methods,the recognition accuracy is increased by 1.72% ～2.55%,achieving better performance of facial expression recognition in natural environment. |
