Research On Controllable Facial Expression Editing Based On Facial Action Unit

Facial expressions do not vary greatly according to human race and gender,and they not only reflect the emotional changes of human faces,but also are an important way for human beings to express their emotions.Therefore,facial expression images have an important role in many fields such as intelligent finance,intelligent security and judicial criminal investigation.However,high-quality facial expression images are scarce and difficult to obtain,so the use of controlled facial expression editing techniques to generate these images has become an important tool for domestic and foreign researchers.The current facial expression editing techniques suffer from poor controllability of the editing process and inconsistent identity of the editing results,which greatly affect the quality of the generated facial expression images.To address these problems,this study proposes a new facial expression editing method that can perform accurate and controllable expression editing while maintaining consistent facial identity information.The main contents of the paper include:（1）In order to keep the identity and content structure consistent when facial expression images are mapped into latent vectors,a new facial expression image mapping model is proposed based on the distribution of facial action units.The facial expression image is divided into several local image blocks to extract features,so that the encoder can learn as many local features of the face as possible.In addition,the first load vector with the largest amount of information is calculated on the feature matrix by principal component analysis as the initial latent vector z₀ of facial expression image mapping.The loss function which can keep the identity information consistent with the content structure is introduced,and the optimal latent vector z^*is obtained by gradient descent method,thus improving the quality of mapping results.Experiments show that compared with the four mainstream algorithms,the mapping model proposed in this paper has greatly improved the loss of content perception and identity information.（2）Aiming at the problems of poor controllability and difficulty in decoupling in facial expression editing,this paper proposes a controllable facial expression editing model based on semantic direction vector.Based on the interpretability of generator latent space,this model combines the principal component of sample matrix and semantic boundary to calculate the semantic vector of expression,thus realizing controllable expression editing.At the same time,a new loss function is designed,which absorbs the content information and structure information of the face image,so that the edited image retains the original content and structure,thus improving the quality of image generation.The experimental results show that compared with other models,the expression images generated by the proposed model in this paper are reduced by 0.47 and 0.25 on the image quality evaluation indexes FID and MMD on average.The image structure and content evaluation indexes PSNR and SSIM are improved by 12.22and 0.0076 respectively.Experiments show that the algorithm proposed in this paper has better controllability and decoupling ability while keeping the identity information consistent.