Vision-based Face Attributes Analysis: Expression Recognition And Face Geometric Perception

In recent years,with the continuous maturity of artificial intelligence technology and the gradual popularization of intelligent scenarios,face attribute analysis has gradually become an important research topic and plays an important role in security monitoring,smart home,human-computer interaction,audio-visual entertainment,and many other aspects.Through face attribute analysis algorithm,the computer is able to extract face semantic information from a raw face image to accomplish many different forms and categories of downstream tasks,helping people to solve many problems encountered in current life through an intelligent way.Face attributes can be divided into non-geometric attributes and geometric attributes in terms of broad categories,where non-geometric attributes include face identity to characterize appearance,face expression to characterize emotion,skin color to characterize ethnicity,etc.,while geometric attributes include head pose to characterize face spatial orientation,3D face model to characterize face shape information,face keypoint location to characterize face parsimonious geometric features,etc.In recent years,face recognition has had relatively more mature development and application,in contrast,some other face attributes still need further research and exploration.This thesis provides an in-depth discussion on the three directions of facial expression recognition,head pose estimation,and 3D face fitting(also known as 3D face alignment)in face attribute analysis,and conducts an exhaustive experimental study on three specific tasks,namely single facial expression image processing,multi-pose facial expression image processing,and multi-task facial geometric attribute analysis,according to different application scenarios and different ways of combining face attributes.This thesis will consider the scenes from simple to complex,from single attribute to multi-attribute,and from theory to application,proposing several innovative solutions for the specific tasks,which effectively improve the accuracy of face attribute analysis.The main innovative work of this thesis is summarized as follows:1.A facial expression recognition algorithm based on region-relationship learning and expression pattern-map generation is proposed.It could pinpoint expression-salient regions utilizing face mask generation and region relationship learning,and could extract discriminative and compact expression features by region fusion modeling.The refined expression features are clustered in the feature space by the metric learning algorithm,which solves the problem of intra-class variability and inter-class similarity in expression recognition.In addition,the method innovatively generates a facial expression patternmap,which can effectively enhance the learning of expression features,improve their discriminative power,and enable visualization of expression features,improving the interpretability of the expression recognition task.A large number of comparative experimental results show that the method can effectively improve the accuracy of the expression recognition task.2.A multi-pose expression recognition algorithm based on pose-aware and heatmap-coupled transformer is proposed.The method addresses the problem of combining facial expressions with head posture by artificially synthesizing an occluded face to simulate the information loss generated by self-occlusion,and training the network to generate consistent expression features for occluded faces,constraining the network to adaptively find unoccluded expression-salient regions.Further,the method uses frontal face images to guide the network to generate pose-invariant expression features through attention relation learning,which improves the robustness of the algorithm.In addition,the face geometry information is used as a priori knowledge to help the network localize the face structure quickly.The experimental results show that the method can effectively improve the accuracy of facial expression recognition in multi-pose scenes.3.A face geometric attribute analysis algorithm based on multi-scale residual merging and frontal face constraint is proposed.The method addresses the problem of combining head pose estimation and 3D face alignment.It realizes hierarchical refinement for the predicted geometric attributes using multi-scale residual merging,which improves the prediction accuracy of the network.At the same time,a frontal face constraint is utilized to unify different geometric attributes into a similar semantic information framework for collaborative training,which helps the network converge to the global optima through multi-task learning.In addition,the cross-attention mechanism and the multi-exit deconfliction mechanism are adopted to ensure the training process to be efficient and reliable,while keeping the overall network lightweight.A large number of experimental results show that this method can effectively improve the accuracy of face geometric attribute analysis.