Learner's Facial Expression Recognition And Sentiment Analysis In Natural Scenes

Facial expressions have rich emotional information and are an important way of expressing human emotions.Facial expression recognition has been applied in many fields as the basis for computer understanding of human emotions.In the learning scene,the emotional state of the learner is closely related to the learning effect.The recognition result based on facial expression can achieve the sentiment analysis of the learner,help to judge the learner's learning state,and then adopt effective teaching and control means to enhance the teaching effect.At present,the research on expression recognition mainly focuses on basic emotions,that is,basic expression recognition,this paper studies the basic expression recognition method based on CNN.At the same time,face images captured in natural scenes have occlusion problems,which may lead to the lack of facial expression information.Aiming at the large-area occlusion face,a GAN-based expression recognition method is proposed.The face image is first filled and repaired,and then the expression recognition is performed.Experiments on the basic expression dataset CK+show that the face image after the restoration is visually true and coherent,and at the same time,a higher expression recognition accuracy is achieved.The study of basic expression recognition laid the foundation for the subsequent learning expression recognition.The emotions that need to be paid attention to during the learning process are different from the basic emotions.Based on the basic expression recognition method,the expression recognition experiment is performed on the learning expression dataset BNU-LSVED.The image-based learning expression recognition is carried out by using the method of fine-tuning depth CNN.On the basis of this,the video-based learning expression recognition is studied further.In addition to the visual features,the temporal features between image frames are used to classify the expressions.Experiments show that VGGface is more suitable as a deep feature extractor for expression recognition than other CNN models,at the same time,after adding the time series features,the video-based learning expression recognition achieves a higher recognition accuracy.Teachers are more concerned with the learning emotional state represented by the learner's emotions.This paper makes an exploratory analysis of the facial emotion recognition results in the corresponding emotional state under the learning scene.Firstly,the Chinese simplified version of the PAD sentiment scale is used to quantify the learning emotions,the three dimensions of the PAD emotional model are analyzed,the corresponding model between the learning emotion and the learning state is established.The model is used to further analyze the learner's learning state according to the result of the expression recognition,including the learner's understanding,interest and engagement to the course content.