Research On Facial Expression And Makeup Transfer Methods Based On GaN

In the field of computer vision,the synthesis and analysis of face images has always been a hot research topic.Facial expression transfer is widely used in film entertainment and artificial intelligence industries,and automatic makeup technology has great application prospects in the beauty industry.Although facial image synthesis technology has been greatly developed in recent years,there are still difficulties and challenges in facial expression transfer and facial makeup transfer: facial expressions vary from person to person,and are the result of changes in facial muscles and textures.It is not easy to naturally and clearly simulate changes in expressions while preserving the original identity;makeup style is always changing and has no rules to follow,it is also affected by age and nationality.Therefore,research and innovation are still needed in the study of facial expression transfer and makeup transfer.In this paper,the author mainly uses Generative Adversarial Networks(GAN)to study the synthesis of face images,including facial expression transfer and facial makeup transfer.The main research work in this paper is as follows:(1)Based on GAN,a multi-expression Generative Adversarial Networks(MEGAN)is proposed.In the network design,MEGAN combines the Variational Autoencoder(VAE),using the coding characteristics of VAE to encode the image carrying the expression information into a latent vector to extract the identity of the face.The emoticon label is integrated into the latent vector,and input to the generator to generate the face image;the discriminator not only distinguishes the true and false images,but also calculates the classification loss of the expression labels,urging the model to output a face image of the specified expression in a label-guided manner;design a set of joint loss functions,including adversarial loss,image reconstruction loss,cycle consistency loss,label classification loss,and Gaussian prior loss.It allows model to output a variety of clear and natural expression transfer images while preserving most of the facial features.Through experimental comparison and analysis,MEGAN is superior to the reference methods in the accuracy of expression transfer and the comparison of partial loss function curves.In the subjective user survey,MEGAN can also get certain recognition.(2)A GAN-based facial makeup transfer method(FMGAN)is proposed,which can perform makeup transformation on face images.This method is an improvement of MEGAN on the network model.The encoder includes a content encoder and a style encoder,respectively encoding the image as a content vector and a style vector,separates the facial features of the image from the makeup style;In the training process,style vectors can be expressed by means of latent regressor to achieve instance-level transformation.The Adaptive Instance Normalization(AdaIN)is introduced to inject the makeup style into the face image to achieve fast makeup transformation;combining face segmentation algorithm and Poisson fusion technology,segmentation-fusion method is used to process and synthesize face images,so that the makeup of detail parts is more refined;due to the continuity of the style vector and the nonlinearity of the generator,the model can automatically output images of different makeup styles.The experimental results show that FMGAN can generate realistic makeup and de-makeup images,which is superior to the reference methods in the color error of makeup synthesis,and can achieve multi-model output.(3)Designed and implemented a virtual makeup system.The system can be used in a variety of makeup retail stores or experience areas,applying the face makeup transfer method to the actual scene,not only for the whole face makeup transfer,but also for the makeup change in the lip area and the eye area.Through this system,users can carry out a good cosmetic trial experience,and merchants can also better sell products and expand customers.