Learning Based Compression Artifact Removal And Face Image Generation With Generative Adversarial Networks

With the rapid development of the digital image processing technology,the information that images express and contain is more and more abundant and diversified.With the wide application of Internet technology,image has become one of the most important and effective information media.However,there exist great challenges in the data storage and transmission due to the huge amount of image data and the limit of bandwidth.Thus,researchers have investigated image compression technology to solve the problems.The purpose of image compression is to reduce redundant information in images so as to achieve the effect of reducing storage overhead.From the mathematical view point,images are represented by two-dimensional vectors and image compression transforms the two-dimensional vector into a statistically unrelated dataset.The transform is usually conducted before image storage or image transmission.Later,the compressed image is decompressed to recover the original image or get the approximate image.This thesis mainly discusses JPEG image format,which is one of the most widely used image compression techniques.Quality factor is a very important parameter,which measures the quality of a compressed image.The higher the quality factor is,the better the image quality is.However,in highly compressed images,i.e.quality factor q ? 10,JPEG algorithm also causes obvious compression distortions including blocking,banding,and ringing,which affect visual perception of the human eyes and are not conducive to subsequent implementing of computer vision tasks such as image classification,object detection and semantic segmentation.Therefore,what the thesis does is to use digital image processing technology to restore the distorted image to the original image as much as possible.On the other hand,image translation has a wide range of applications.Among them,the generation of facial images is one of the issues that people are most concerned about with image translation technologies.At present,most methods are based on conditional generative adversarial networks(c GANs)with U-Net as the backbone.They need auxiliary information such as segmentation maps and parsing maps to generate visually pleasing facial images from sketches.However,it is difficult to obtain these semantic information in reality.So,we consider a very practical situation where people can only draw a simple facial sketch to describe an imaginary person.There is only partial information of human faces in portraits.Therefore,generating real human faces only from sketches is a challenging problem in computer vision and graphics.Therefore,we assume that we just only use rough facial sketches as input for training our proposed network,which is very consistent with realistic conditions.This thesis investigates compression artifact removal and face generation based on deep learning.In short,compression artifact removal belongs to the field of image restoration,which transforms the distorted image with high compression rate into a clear image with good quality and rich textures.Face generation belongs to the field of image translation,which transforms a rough facial sketch into a photo-realistic facial image.The main contributions of this thesis are as follows:1.For the compression artifact removal task,the thesis proposes a weakly connected dense generative adversarial network,called WCDGAN.The generator of WCDGAN consists of three main ingredients: mixed convolution,weakly connected dense block and attention mechanism module.Firstly,the thesis produces a mixed convolution module composed of standard convolutions and dilated convolutions,which enables the network to capture a larger receptive field and reduce the grid effect caused by dilated convolution.Secondly,the thesis provides a weakly connected dense block(WCDB),which can add shallow features to a deep network for feature reuse,balance memory consumption and computational efficiency.Thirdly,the thesis combines channel attention and spatial attention with mixed convolution into WCDB to capture features with strong information expression capacity.Finally,the thesis adds a perceptual loss to make the restored images more realistic.The experimental results show that WCDGAN can successfully remove the blocking and banding compression distortions,and produce sharp edges and clear textures even from highly compressed images.Moreover,WCDGAN outperforms state-of-the-art methods for compression artifact removal in terms of peak signal-to-noise ratio(PSNR)and structural similarity(SSIM).2.For the face generation task,the thesis proposes a novel multi-level and hierarchical generative adversarial network.It divides a complete task into many sub-tasks,and allows the sub-networks to learn features with different scales.The proposed network does not require any auxiliary information for face generation,but only need a rough sketch as input.The thesis uses the previous-level generator to generate facial features,then concatenate them with the down-scaled input sketch,and put them to the next-level generator together to produce sharp lines and completed facial contours.Corresponding to the multi-level network structure,the thesis designs a multi-scale loss function to constrain the parameters of the multi-level network,thus leading to stable convergence.The multi-level GANs progressively generate fine textures and contours in facial images,resulting in photo-realistic facial images even from rough sketches.Various experiments show that the proposed network generates natural-looking facial images and outperforms state-of-the-art methods in terms of both visual quality and quantitative measurements.