Study On Facial Expression Recognition Algorithms Based On Deep Learning And Feature Fusion

In recent years,with the rapid development of deep learning and artificial intelligence technologies,people have put forward higher demands on human-computer interaction,leading to facial expression recognition becoming a research hotspot,due to the ability to provide more accurate and personalized interaction through expression recognition.Current facial expression recognition algorithms are divided into traditional algorithms and deep learning-based algorithms.Traditional facial expression recognition algorithms require human feature extraction method design,while deep learning-based algorithms are based on neural networks and build end-to-end network models to automatically perform feature extraction,which can approximate arbitrary nonlinear functions with arbitrary accuracy,thus becoming the mainstream research direction at present.Although the accuracy of face expression recognition methods using existing deep learning can reach or approach 100% in the laboratory state,they cannot be applied to real environments yet.Therefore,in this paper,with the main research goal of improving the recognition accuracy of facial expressions in real-world,we explore and study the face expression recognition algorithms based on deep learning and feature fusion in real environments,construct two face expression recognition algorithms based on convolutional neural networks,and design and develop a face expression recognition system.The specific work and research contents are as follows:(1)For the problem that each part of the face contributes differently to the expression and the general algorithm cannot extract the local features of the face,a face expression recognition algorithm based on the fusion of global and local features is proposed,in which the local features of the face are extracted through an independent network,and feature fusion is performed twice before the overall image is input to the network and after the global features are extracted,respectively,and the channel attention module is used to The corresponding weights are applied to different channels using the channel attention module.Among them,for the extraction of local features,a test network is constructed for experimental comparison between the two approaches of segmenting the image and intercepting the eye and mouth regions of the face,while the 9-segment,16-segment and 25-segment of the segmented image methods are further compared experimentally.Finally,a multi-scale convolution module is added to the overall network architecture,which is composed of convolutional layers with different sizes of convolutional kernels in the same convolutional layer and pooling layers,in order to extract and fuse features of different scales,and to reduce the number of parameters,a channel dimensionality reduction operation is performed before extracting features of different scales,and the channel dimensionality is recovered by a splicing operation.The performance of the algorithm is verified by comparison experiments and ablation experiments on the FER-2013 dataset and RAF-DB dataset collected under non-laboratory conditions,and the final accuracy reached 72.03%and 87.32%,respectively.(2)To address the problem that relying on a single loss function in deep learning neural networks cannot take into account the optimal weight parameter updates of both shallow and deep networks,a segmented network and attention mechanism based face expression recognition algorithm is proposed,which divides the overall network into a shallow feature extraction part of the main network,and a deep feature extraction part of the attention module,and back propagates through different loss functions.Where the attention module part crossentropy loss function,the main network part uses center-angle loss function,which uses center loss to reduce intra-class distance while using angle loss to expand inter-class distance.The attention module is a series-parallel two-row network consisting of a multidimensional attention module and a multiscale attention module: the multidimensional attention mechanism weights all three dimensions of the input features,where the channel dimension is put through a separate network layer for the extraction of the weighting coefficients due to its own specificity,while the parameters of the other network layer are shared and the extraction of the weighting coefficients of these two dimensions is performed due to the similarity of the width and height dimensions.The multi-scale attention module does not choose the common linear combination between the channel attention module and the spatial attention module,but adds a multi-scale convolution module and a residual structure between the two attention modules to make up for the shortcomings of the traditional attention that cannot extract multi-scale features and that the upper and lower layers are not tightly connected.Finally,the features extracted from the convolutional layer are downscaled by using the weighted downscaling method,so that the original features can be utilized reasonably and adequately under the control of dimensionality.The algorithm is validated by comparison experiments and two kinds of ablation experiments on FER-2013 dataset and RAF-DB dataset,and the final accuracy reached 73.22% and 89.05%,respectively.(3)Design and development of an embedded facial expression recognition system:based on hardware devices such as jetson Xavier NX edge computing device,camera and display,the face facial expressions are recognized using the model trained by the face expression recognition algorithm proposed in this paper,and Pycharm software is used to design and develop a facial expression recognition system with a graphical user interface using modules such as Py Qt5 and Open CV.The system is capable of recognizing facial expressions with a graphical user interface.This system can meet the needs of face expression recognition for image files as well as real-time face expression recognition by camera.In this paper,we conduct an in-depth study on the face expression recognition algorithm in real environment,and propose a face expression recognition algorithm based on the fusion of global and local features and a segmented network and attention mechanism,and conduct experimental validation by FER-2013 datasets and RAF-DB datasets.And we designed and developed a face expression recognition system based on jetson Xavier NX edge computing device.The research of this paper has a certain promotion effect on the theoretical algorithm and practical application of face expression recognition,and lays a good foundation for the wide application of more humanized human-computer interaction.