Research On Image Synthesis Methods Based On Generative Adversarial Networks

Image synthesis is a technology that manages random noises,labels,or semantic features to learn how to synthesize the target image.Recently,as one of the significant research topics in the scope of computer vision,image synthesis is widely utilized in various applications,such as image editing,film effects,and so on.However,due to the high diversity and complexity of natural images,it is still a great challenge to build a model for synthesizing high-quality samples.In 2014,the proposed of Generative Adversarial Networks(GAN)matches the demand for deep learning and promotes the development of image generation.Nowadays,the research of image synthesis based on GAN has been widely concerned by scholars in related fields.Although some existing GAN models can synthesize samples that look realistic,they still suffer some problems due to the setting of network structure,the characteristics of training data-sets(diversity and complexity,etc.),and optimization strategies.For instance,the GAN models always share problems of training instability and mode collapse during training;the samples synthesized by the GAN models have the problem of missing details due to the lack of capturing detailed information.We find that if the discriminator of GAN can learn features with powerful representative ability during training,it will help to improve the property of the model.The successful application of the deep metric learning algorithm as a classical method to extract representative features in classification tasks drives us to apply it to the GAN model to improve the quality of synthesized samples.Besides,due to the lack of direct constraints on the detailed features of samples,the synthesized images are relatively roughly.The unique localization properties of the Two-Dimensional Discrete Wavelet Transform(2D-DWT)algorithm inspire us to apply it to GAN to improve the detailed integrity of the synthesized images.In this paper,the deep metric learning method and the 2D-DWT algorithm are employed to enhance the property of GANs in synthesizing samples rich in details by re-changing the objective function and model structure.The main works are summarized as follows:(1)We propose an image synthesis method named Discriminative Metric-based Generative Adversarial Network(DMGAN).To improve the performance of GAN and enhance the quality of synthetic images,we take advantage of the characteristic of deep metric learning to learn the compactness features within classes and separability features between classes.Specifically,we conduct the discriminator of our model as the form of feature extractor so that the discriminator can extract features with representativeness by the joint optimizing of identity preserving loss and discriminative loss.Meanwhile,we train the generator to synthetic real-like samples by reducing the distance between the features of real and generated ones.Finally,the comparative experiments on several classic databases illustrate that our model share advantages in the quality of synthesized samples compared with the popular image synthesis methods.(2)We propose an image synthesis method named Weight-adaptive Discriminative Metric-based Generative Adversarial Network(Wa DMGAN).If the discriminator of the metric-based generative adversarial model can not acquire features with strong representability.It is hard for them to distinguish real samples and fake ones accurately.Then the generator will synthesize fake images that share enormous gaps from real ones.The main idea of Wa DMGAN is to promote the performance of the discriminator in DMGAN and enhance the quality of the generated samples.Specifically,keeping the optimization strategies in DMGAN,we additionally supplement a data-dependent weight adaptive hyper-parameter to the discriminator,which will promote the discriminator to pay more attention to the pairs of true and false samples that are difficult to distinguish and reduce the energy on easily distinguishable samples.Finally,by comparing with other models in classical datasets,it is verified that Wa DMGAN further enhances the quality of the generated samples.(3)We propose an image synthesis method named Discrete Wavelet Transformbased Generative Adversarial Network(DWTGAN).In the GAN models that only focus on the image domain features,the images synthesized by the generator are often smooth due to the lack of a direct cover of the detailed features.To enhance the detailed features of the synthesized image,DWTGAN utilizes the advantages of the 2D-DWT algorithm in extracting features of images in the frequency domain and tries to synthesize and distinguish the original image and each sub-band image in the frequency domain,respectively.To be specific,the generator consists of three sub-generators.Among them,one subgenerator synthesizes regular images by learning features from the image domain,and the other two sub-generators learn features from the low-frequency and high-frequency characteristics,respectively,and obtain details of images through inverse wavelet transformation.The final images synthesized by DWTGAN are the fusion of regular rough samples and detailed samples.Meanwhile,the discriminator distinguishes real samples from false ones by constraining both frequency domain features and image domain features.Qualitative and quantitative results illustrate the superiority of our model over other state-of-the-art models.