Learner Expression Recognition And Application Based On Interpretable Hierarchy Contrast

The imbalance of emotional interaction between teachers and learners in the classroom seriously affects the interaction and effectiveness of the classroom.As one of the important ways to reflect the emotional expression of learners,learners’ facial expressions contain the change process of learners’ inner emotions during the learning process,which provides a way to solve the problem of learners’ emotional imbalance.However,current facial expression recognition models only focus on improving the features of expression itself,and ignore the features that can distinguish among facial expressions and the features confused among facial expressions,which are often the key factors to improve the efficiency and robustness of facial expression recognition.Moreover,most of the current facial expression recognition models are not interpretable.In view of this disadvantage,these models make the reduction of robustness when they are applied to learner facial expression recognitions in classroom scenarios.In order to solve the above problems,this paper proposes a learner expression recognitions model based on interpretable hierarchy contrast.The main contents are as follows:1)We propose a facial expression recognitions technology based on cross-hierarchy contrast.By contrasting local facial features from multiple hierarchies,we can obtain the differential and the common features,and design a fusion network to reduce noise interference.We conduct experiments on three public expression datasets: CK+,FER+ and RAF-DB,which can verify the effectiveness of the model;2)We propose an interpretability mechanism based on cross-hierarchy contrast model,and elucidate the expression recognitions process based on the decision tree.Moreover,this study uses transfer learning to realize the model prediction of small sample data in real classroom,and conducts learners’ facial expression data collection and experiments for the online learning environment,which verifies the validity and practicability of the model proposed in this study;3)Relying on the iStudy online learning platform,we designed a learner’s emotional problem diagnosis system.The system is oriented to the online learning environment.While acquiring the emotional of the learners,it presents the expression categories and facial local features corresponding to negative emotions.In this way,teachers can check the correctness of the emotional results,and analyze the reasons for negative emotions in combination with current knowledge points,which can realize the diagnosis of learners’ emotional problems.In this way,teachers can provide personalized interference and guidance to learners to enhance the interactivity and efficiency of the classroom.Aiming at solving the problem of learners’ expression recognition and corresponding emotional state analysis,the innovation of this study is mainly emphasized on:(1)A crosshierarchy contrast mechanism is designed to obtain the key features of expression recognition and the features of confused expression recognition in pursuit of improving the accuracy and efficiency of expression recognition model.(2)The interpretability and visualization mechanism provide guarantee for the design of visualization system.(3)The application of the diagnosis of learners’ emotional problems based on the i Study platform provides support for teachers to grasp the learners’ negative emotions in a timely manner,effectively confirm them,and analyze the interpretability of emotional problems.