| Human facial expression recognition(FER)technology not only has important applications in many fields such as healthcare,education,policing,retail,transportation and services,but also can help humans communicate more effectively with intelligent machines.Facial expression recognition technology is mainly used to analyze human emotions with the help of computer tools,so as to determine the real inner activities and emotional states of human beings.In the past two decades,with the explosive development of deep learning technology,how to analyze human face expressions more accurately and quickly has become a very popular research content in the current computer vision field.However,previous work has rarely used finer-grained facial action units(AU)for human facial expression analysis tasks,which to a certain extent limits the accuracy of expression recognition.For this reason,this thesis attempts to investigate facial expression recognition from the perspective of AU,expecting to further improve the recognition accuracy of expression categories.The specific research contents and contributions of this thesis are as follows:(1)This thesis propose a multi-feature fusion method for predicting the intensity of AU.The method also takes into account the different ranges of activities of individual AUs and the dependencies that exist between AUs,and then a multi-level multi-scale local area relationship attention model is used to obtain AU spatial area relationship features.The method fuses the captured AU spatial region relationship features,facial geometric features,and global deep features of the face to achieve a more accurate prediction of AU occurrence intensity.In this thesis,the feasibility of the method is verified on the publicly available DISFA dataset,and the experimental results show that the method outperforms some current mainstream methods.(2)This thesis designs a method to enhance expression category classification based on the occurrence region and occurrence intensity of AU,which can enhance the discriminative nature of expression features to some extent.First,the method calculates the local region and the location of the activity center of each AU with the help of face feature points.Then,in this thesis,the local region of each AU is used to learn potential relationship features between AUs,and the intensity of AU occurrence at each AU activity center point is used to enhance the performance of face appearance in an approximate Gaussian distribution.Finally,in this thesis,the captured AU potential relationship features and the enhanced global expression features are used for expression category recognition.To verify the effectiveness of AU-enhanced expressions,this thesis conducts experimental validation on several expression datasets and proves the effectiveness of the method.The method also achieves a high level of recognition compared with current excellent methods.(3)Considering that the method of AU-enhanced expression recognition can adequately capture the relevant content of human face AU and can combine multiple other facial information of human face for more accurate expression analysis tasks.Therefore,this thesis attempts to apply the method to the analysis of autistic and nonautistic facial expressions in children,hoping to provide an effective technique for diagnosing children with autism.In this thesis,an experimental study is conducted on an available dataset of images of children with autism,and the experimental results show that the method can effectively perform expression classification of autistic patients.At the same time,this thesis further validates the effectiveness of AU occurrence intensity enhancement expressions on this dataset. |
