Image Synthesis And Quality Assessment Via Deep Learning

Image Generation is an important research direction in the fields of computer vision and computer graphics.Currently,this field has been studied for more than 70 years.Early generative models can only generate simple,low-quality images.Recently,with the rapid development of deep learning,image generation has made great breakthroughs and is able to generate high-quality,controllable,and diverse results.It has been widely used in today’s society,such as:image enhancement,image editing,cross-modal image generation,image style transfer,movie special effects,art design,and so on.Different application scenarios often require different generative models.Therefore,choosing a suitable generative model for different applications is very important.In addition,since the generative models inevitably produce some low-quality images,how to objectively and fairly evaluate the quality of the generated images and select high-quality images is also a very important research problem.This thesis provides in-depth analysis and solutions for these challenges,the main contributions include the following four aspects:First,this thesis proposes an optimization based deep feature reshuffle for image style transfer.Previous works could only transfer global style or local style.By reshuffling the deep semantic feature of the style image according to the content image,then generate the style-transferred image through the decoder.This algorithm can be theoretically proved to transfer global and local style information at the same time,and experiments prove that this algorithm can produce high-quality images for various styles.Second,this thesis proposes a portrait editing algorithm based on conditional generative adversarial network.This method uses the portrait mask obtained by the face segmentation network to disentangle the mask and the face attributes,by encoding different parts of the face separately,it could achieve high-quality,diverse and controllable portrait editing.At the same time,the framework can be applied to many applications,such as mask rendering,face swapping with hair,and using the generated image as the data augmentation for training a segmentation network.Third,this thesis proposes a new generative model,Vector Quantized Denoising Diffusion Model.This method converts natural images into discrete codes by using a VQVAE,and then uses a discrete denoising diffusion model to generate reasonable codes,and decode them to obtain a generated image.It can achieve strong performance on many tasks such as text-to-image generation and unconditional image generation.On the one hand,compared with the Auto-Regressive models,this algorithm is more efficient and avoids the problem of uni-directional bias and error accumulation.On the other hand,compared with the generative adversarial network,the algorithm has a stronger capability to capture more complex data distribution.Last,this thesis proposes a new research topic:Generated Image Quality Assessment.This topic aims to study how to automatically,objectively evaluate the quality of the generated images.Based on it,three generated image quality assessment algorithms are proposed,and a dataset is collected to verify their performance.Furthermore,this thesis explores the great value of generated image quality assessment,such as generative model assessment,improving generative model performance,filter high-quality images.