Research On Face Image Editing Based On Prior-aware Deep Adversarial Learning

Face image editing has become one of the most popular research topics in the field of computer vision in recent years.It encompasses various research problems such as facial pose editing,makeup removal,facial attribute editing,and facial swapping.With the rapid development of deep learning and the continuous improvement of computing power,a large number of face image editing technologies have achieved vigorous development and have been widely applied in people’s daily lives.These technologies have high value in terms of both academic research and industrial application.Due to the unique physical and semantic characteristics of faces,existing face editing methods often incorporate simple face prior knowledge(such as facial landmarks,semantic maps,or 3D facial information)to improve editing performance.Although some progress has been made,existing methods still face numerous challenges.For example,current face pose editing methods suffer from relatively weak deformation modeling capabilities,making it difficult to achieve identity-preserved large-pose face transformations? existing makeup removal methods have limited representation capabilities and cannot explicitly perceive makeup semantics,resulting in imprecise makeup removal and failure to restore occluded facial details? current textdriven facial attribute editing methods are limited by the model’s inherent learning ambiguity and inaccurate supervision signals.They are unable to learn accurate attribute editing directions,resulting in poor editing performance and semantic consistency.Built upon the Generative Adversarial Networks(GANs),the purpose of this thesis is to achieve high-quality,precise,and controllable face image editing.The focus lies on how to better utilize face-related prior knowledge to guide model learning.Specifically,this thesis explores three key sub-tasks in face image editing: face pose editing,makeup removal,and text-driven face attribute editing.Several innovative contributions and achievements have been made in the following aspects:To deal with the challenges of inadequate modeling complex facial deformations and poor identity preservation under large pose variations in existing methods for face pose editing,a novel framework based on face deformation priors is proposed to achieve precise modeling of facial deformation under large pose changes,which improves the effectiveness of large pose editing and identity preservation.It consists of an personalized landmark learning module and a gated deformable face synthesis module.The former transforms the input reference landmark into individual-specific pose information based on the source facial structure,providing more accurate pose guidance for subsequent face generation.The latter explicitly models and samples complex facial deformations caused by pose changes using the proposed gated convolutional blocks,ensuring accurate face pose editing under large pose variations.In addition,the potential of skip connections in face pose editing is explored.A novel pose-aligned skip connection is proposed to directly propagates the low-level facial details of the source face image to the correct pixel locations on the output image,effectively improving the model’s ability to maintain identity under large pose variations.Extensive experiments on six constrained and unconstrained face datasets demonstrate that the proposed method outperforms existing approaches in both large pose editing and identity preservation,while also demonstrating good generalization ability.To address the issues of the existing methods being unable to precisely locate,remove makeup,and recover occluded facial details,a facial makeup removal network based on semantic-aware generative prior fusion is proposed.This network consists of a makeup prediction and highlighting module that utilizes multi-level makeup responses to automatically locate and estimate the distribution and degree of makeup? a generative prior module that provides rich and diverse facial image representations? and a makeup removal module that integrates generative prior features with makeup-removed facial features to generate high-quality makeup removal results.To leverage generative prior features to facilitate accurate makeup removal and facial detail recovery,a semantic-aware feature fusion block is designed to effectively inject generative prior under the guidance of makeup semantics.Experimental results on both synthetic and real facial makeup images demonstrate that compared to the existing methods,the proposed method can remove makeup more accurately and restore realistic facial details.To solve the problem of insufficient accuracy in learning attribute editing directions for current text-driven face attribute editing models,this thesis proposes a text-driven face attribute editing network based on semantic correlation learning of latent directions.To reduce the ambiguity in model learning,a latent direction learning and combination module is designed,which learns an orthogonal latent direction library shared by different attributes,and predicts the final semantic editing direction by linearly combining the latent directions.This design can effectively utilize the semantic correlations among different attributes,reduce the difficulty of model learning,and improve the accuracy of attribute editing direction prediction.Additionally,to obtain more sufficient and accurate supervision information for network optimization,a multiattribute editing direction contrastive loss is proposed,which maximizes the mutual information of the same attribute editing direction in the CLIP(Contrastive Language-Image Pretraining)space and minimizes the mutual information of different attribute editing directions.This comprehensively exploits the language-vision cross-modal face prior knowledge of CLIP and provides more precise supervision signals for accurate learning of editing directions.Extensive experimental results demonstrate that the proposed method is superior to current methods in both face attribute editing quality and semantic consistency.