| Online-education is developing rapidly due to irreplaceable convenience,which is widely available around the world.Recently,numerous schools have delayed opening caused by COVID-19,which made online-education be one of the primary teaching methods.However,compared to traditional face-to-face classrooms,the effectiveness of online courses has been questioned due to lack direct,timely and effective communication and feedback between teachers and learners.As a result,the problem of "emotional loss"is widespread in online learning environment.Research has suggested that emotion of learner is closely linked to the learning status.In addition,the emotional state will seriously affect their knowledge acquisition and overall goals.Therefore,the importance of emotional factors cannot be overlooked,and how to address the problem of"emotional loss" in online-education is a hot research topic in psychology,education and computer science.Furthermore,previous research has revealed that a close and stable relationship is existed between facial expressions and emotions.In this paper,in order to solve the problem of "emotional loss" in online-education,facial expression recognition is used to perceive learner emotions and realize real-time feedback of emotions in online education.It not only helps to strengthen the interaction between teachers and learners,but also helps teachers change the teaching strategies in the online learning environment according to the emotional changes of students,and promote personalized education.Overall,the principal work of this research is as follows.(1)Construction of facial expression recognition model based on Gaussian prior distributionExpressions are formed by local muscle deformation of the face,and different expressions often have the same or similar local muscle deformation.Based on a large number of studies and analysis of expressions,there is an implicit similarity between among expressions.In addition,the similarity of each expression with other expressions follows the Gaussus-like distribution sorted by a certain rule.It is also noteworthy that expressions are often more than one emotion,but a mixture of emotions.Based on the above observes,the emotion label distribution based on Gaussus distribution is designed to replace the traditional hard label.Based on the emotion label distribution of each expression,a real-time expression recognition model using deep convolutional neural network is constructed.And the model is lightweight and optimized.The KL divergence is utilized as the loss function,which can more effectively measure the distance between two distributions.In addition,the L2 normalization is employed to balance the performance and complexity of the model.Experimental results on published facial expression datasets demonstrate that the proposed algorithm has achieved the superior performance.It would save time and calculation cost as well as improves performance and robustness.(2)Design of graph neural network and representation learning method for facial expression recognitionA novel model for facial expression recognition in-the-wild datasets is proposed to suppress the uncertainty of facial images in the real environment.The uncertainty may result from the ambiguity of expression,the subjectivity of the annotator and the low-quality facial images.The overview of the proposed method is as follows.Firstly,the facial images are grouped into high and low uncertainties by pre-trained network.The graph convolutional network framework is utilized for the facial images with low uncertainties to obtain geometry cues.These cues include the relationship among action units(AUs)and the implicit connection between AUs and expressions,helping to predict latent emotion label probability.The emotion label distribution is subsequently produced by combining the prediction latent label probability and the given label.The k nearest-neighbor graphs are built for the facial images with high uncertainties to determine k facial images in the low uncertainties group with the highest similarity of given a facial image.The emotion label distribution of the given image is then replaced by fusing the emotion label distributions according to the distances between the given image and its adjacent images.Finally,the constructed emotion label distribution facilitates training in an end-to-end manner by a convolutional neural network framework to identify facial expressions.(3)Real-time acquisition and analysis of learner’s emotionsA real-time emotion analysis mechanism is built for the learner.Firstly,the video data of learning is obtained through the camera.Then the video data is sampled,and each frame of image is taken as subsequent input.In order to reduce the interference caused by background elements,face recognition is performed on the sampled image.Specifically,the ROI pooling method is utilized to select the face candidate box,the image containing face area and then is cropped.The facial image is input pre-trained expression recognition model to get the learner real-time expression state.The expressions are further divided into three categories of learning emotions including positive(happy,surprise),negative(sadness,fear,angry,disgust),and neutral(neutral).Experimental results show that the model can obtain the emotions of the learner objectively and accurately in real time. |
