Research On Prediction Methods Of Continous Emotion Dimension Based On Human Face

In the field of sentiment computing,with the development of machine learning and artificial intelligence,the research on the continuous emotional dimension prediction combined with psychology and neurophysiology is also rapidly advancing.Based on the dimensional theory of emotions,the continuous emotional dimension prediction is no longer to describe human affective as a discrete,single class,but to study the continuity of human emotions expressed naturally.As a result,the research has become closer to the natural expression of emotions in human daily life.And there are three main aspects of its research: database construction,feature extraction,and regression prediction model construction.This paper focuses on the emotion dimension prediction based on the facial dynamic feature extraction scheme.And the main work includes the following three aspects:(1)In order to thoroughly understand and study the continu ous emotional dimension prediction problem,its research background and related theories are firstly summarized in this article.And the methods of continuous emotional dimension prediction published in recent years are also summarized,and systematically analyzed and compared as the same time.(2)In view of the lack of previous research on the influence of time series orientation on the ability to predict continuous emotional dimensions,we first analyze the LBP-TOP feature extraction scheme for historical direction and future direction.And then,based on the analysis results,a dynamic emotion feature extraction scheme named Bi-LBP-TOP(Bidirectional Local Binary Pattern Three Orthogonal Planes)is designed.First,a sequence of image frames for an appro priate time step is read from a video clip,and face detection and cropping processing are performed.Then,Bi-LBP-TOP is used to extract temporal dynamic emotion features.Finally,The experimental results based on the AVEC2012(FCSC)dataset show that th e Bi-LBP-TOP scheme is more stable than historical and future LBP-TOP scheme when the time step changes,and its overall performance is significantly higher than the benchmark experiment using LBP feature extraction method.(3)A Facial landmark points based Bi-LBP-TOP(FLP-BLT)is designed,which can weaken the influence of face pose change on the extraction of facial dynamic emotion features.The FLP-BLT is able to align the key areas(eyes,nose,and mouth)of facial emotion information expression in the time-series direction.Thereby,more reliable and continuous dynamic contextual emotion-related information in the facial image sequences can be mined.Firstly,a sequence of image frames for an appropriate time step is read from a video clip,and face detection and cropping operations are conducted whereafter.Then the discriminative response map fitting algorithm is executed to detect facial landmark points,and an appropriate size block is cut with each feature points of the key areas as a center.And,for the corresponding block sequences of these blocks,the Bi-LBP-TOP is utilized for gaining temporal context related dynamic emotion information.Finally,The experimental results based on AVEC2012(FCSC)dataset show that the performance of FLP-BLT in this paper is significantly higher than that of the benchmark experiment.