Expression Recognition Based On Multi-scale Feature Fusion In Classroom

With the development of "Artificial Intelligence + Education",the use of expression recognition technology to identify learners’ emotions has become the most direct and effective method.Facial Expression Recognition(FER)has received widespread attention in recent years as an important component of computer vision and intelligent human-computer interaction.The feature representation of facial expressions directly affects the recognition effect,and traditional feature extraction methods are difficult to extract the deep features in human face expressions,while deep convolutional neural networks can obtain features that are difficult to extract manually,but there are drawbacks that consume a lot of computing time and memory.As a result,complex classroom environments place more demanding demands on effective models.Aiming at the above problems,this paper studies the facial expression recognition method based on deep learning.Firstly,the principles and basic modules of typical convolutional neural networks in deep learning are introduced in detail;secondly,the problems and shortcomings in the current facial expression recognition methods are enumerated;the Multi-scale Residuals Networks(MS-RNet)model is proposed to achieve higher precision expression recognition on open source datasets;and the proposed multi-scale feature fusion ideas are used to improve the residual network.Experiment on a self-built classroom student expression data-set and design a system to implement automatic facial expression recognition.The main tasks are as follows:For most of the existing facial expression recognition models,they only show good results on specific data sets,and the depth model has the problem of high computational complexity due to the large number of parameters.In this paper,an end-to-end multi-scale residual network model is proposed.This model extracts multi-scale features(global features and local features)from the face image of the face,and realizes the fusion representation of global features and local features by using channel merged feature fusion method to achieve high-precision face expression recognition.In addition,the use of separable convolutions instead of original convolutions achieves the purpose of increasing the depth of the model without increasing the parameters of the model.In order to verify the validity of the proposed model,we trained and tested on the open source datasets CK+,FER2013 and JAFFE.Experimental results show that the proposed model can achieve a high recognition effect.Aiming at the problem of facial expression recognition in natural classroom scenes,based on the proposed multi-scale feature fusion idea,a multi-scale fusion residual network model is designed.By connecting the residual results of different scales,the model realizes the fusion of multi-scale features for facial expression recognition,which effectively suppresses the partial occlusion problem in classroom scenes and improves the recognition accuracy.Using the video data obtained from video surveillance in campus classrooms,we built a data-set of student expressions in natural classroom scenarios,and conducted training and testing on this data-set.The results show that the average recognition rate of the constructed neural network model on the data-set reaches 93.96 %.The experimental results show that the method can basically meet the practical application.Through the analysis of the actual scene,the design system realizes the automatic facial expression recognition,which will provide strong data and technical support for the follow-up research on students’ classroom learning evaluation.