Face Image Synthesis Method Research And Application Using Machine Learning Based Image Generation Algorithm

Along with the rapid development of the calculation ability of various electronic equipment over the past decades,especially the parallel calculation ability enhance-ment of the general purpose graphic processing unit(GP-GPU)of recent years,machine learning technics had been rapidly developed.After the appearance of convolutional neural network(CNN)based deep learning methods,traditional hand-craft mathemat-ical feature based methods had been overtaken,especially after generative adversarial network(GAN)based image processing methods appeared since 2016,beyond tradi-tional image recognition,image improvement,and image segmentation methods,prior knowledge based image synthesis methods also been invented.But the synthesis results still have the problems of hard to converge,massive calculation,slow optimization rate,image degration problems,et al.For face image synthesis,the existing algorithm mainly focus on the image quality and detail pattern synthesized results instead of identification information and the similarity of the source image and synthesis reuslts which should be research and resolve.This thesis mainly focuses on the GAN based facial image synthesis,which was divided into 3 phases.Firstly,the segmentation of main targets in the image;secondly,facial image synthesis was executed;at last,image quality was enhanced and the super-resolution method was applied,the whole pipeline of image recognition,segmentation,synthesis,and quality enhancement was realized.The main work and highlights of this thesis include:1.Through the Gaussian mixture model(GMM)clustering on statistical color information,a fast image segmentation method,which can segment an image into target areas by hierarchical bisections and get the corresponding target outline simultaneously,was developed.This method,different from traditional contour to continuous outline then to target segment area methods,got the target area segmentation from statistical color information and morphological pixel distribution directly,omitted the continuous outline calculation,and get the outline from segmentation area straight,which saved the calculation time and got robust segmentation results.2.Through facial area segmentation,background bokeh,and filling missed pixels,GAN based standard facial image synthesis method was developed,the synthesized image results were aligned by facial landmark points and have normalized synthesis mode on the face,shoulder,and the upper body part.Standard profile images including important facial feature information were generated,and the identification information was kept from the original input image.This method can be used as the post-procedure of face recognition methods,and make the recognized area into synthesis results with normalized image pattern and aligned face area;it also can be used as a pre-procedure of facial recognition methods and segmentation methods to improve the algorithm results.3.Finally,this thesis developed a facial image super-resolution method which is optimized for photo-realized synthesis results as well as keeping the identity informa-tion of the original input image and got a high-resolution result with 4 or 8 scale super-resolution factor with carefully designed architecture combined with GAN and residual network,which can synthesize the blurry input images into high-quality output images with high resolution and photo-realistic facial detail.This method has well generic and robust features in different datasets and adopted an end-to-end flexible hierarchi-cal structure that fits for multiple algorithm optimization targets and image synthesis demands.4.In evaluation part,this thesis adopted Frechet Inception Distance(FID)as the image synthesis and super-resolution result evaluation benchmark,different from the traditional peak signal-to-noise ratio(PSNR)and structural similarity index(SSIM)value,the FID value can evaluate the facial image synthesis results by considering the global statistical information and express the facial identity information more precisely.PSNR and SSIM value,on the other hand,focus more attention on the local feature de-tails of the image and are used by methods to evaluate the image detail quality,image sharpness,and local information.By combining these 3 evaluation standards,the syn-thesized facial image quality and identity information from the original input image can be evaluated at the same time.