Detail Transfer Of Fine-Grained Facial Makeup

The purpose of makeup transfer is to transfer the makeup style of the reference image to the non-makeup source image while preserving the face identity of the source image.It is a frontier direction that has been widely concerned in the field of deep learning at present,and has strong application value,including Beauty cosmetics software one-click makeup,virtual makeup test of cosmetics,Video makeup transfer(makeup on live broadcast software,makeup on TV characters),etc.Recent research shows that when the facial pose and expression difference between the source image and the reference image is too large or the reference image has artifacts,the makeup transfer result image will have artifacts and the makeup color cannot be transfered correctly.The reason is that the handling of fine-grained makeup details transfer is not perfect.Different from the previous makeup transfer method,this thesis combines the pyramid feature hierarchy structure and the dual attention module on the basis of the generation adversarial network(GAN),so that fine-grained makeup details can be better transferred.In detail,firstly,we use Makeup Extraction Network(MENet)to extract multi-resolution makeup matrix of reference image;Secondly,the makeup matrix of the reference image is transformed into the makeup matrix of the source image by using the Attentive Makeup Morphing(AMM)module;Lastly,the makeup matrix is integrated into the source image by using the Makeup Integration Network(MINet),so as to obtain the final makeup transfer result.Finally,both qualitative and quantitative experiments show that the method in this thesis has achieved better results in fine-grained makeup details transfer.In addition,through ablation studies,the effectiveness of dual attention module and pyramidal feature hierarchy in transferring fine-grained makeup details was verified.In this thesis,a more accurate model is used to make the makeup transfer result figure have a better visual perception effect.At the same time,in order to verify the practicability of the fine-grained makeup transfer model applied to virtual makeup trial system,considering the actual makeup test situation,users may want to view their own makeup transfer renderings under different expressions.Therefore,this thesis first generate a smile image through the face editing model STGAN,and then use the fine-grained makeup transfer model to output the makeup transfer effect map.The experimental results show that the fine-grained makeup transfer model can generate high-quality makeup try-on renderings under different expressions of the same person,which means that the fine-grained makeup transfer model has high practicability in the actual makeup try-on scene.This is of great significance in improving users’ satisfaction with makeup fitting and promoting the company’s beauty products.