Generative Adversarial Network Optimization Strategy Research

Generative Adversarial Network(GAN)has been widely used to generate a lot of simulation data to supplement insufficient real data,and it has been applied to many real-world applications.However,GAN still exists challenge which causes that generating high quality simulation data is a non-trivial task.Although GAN variants,such as Wasserstein GAN(WGAN)and Multi-generator GAN(MGAN),have been proposed to address these challenges,there are still challenges.Existing GAN variants either produce a negative gradient during training(e.g.,WGAN)or result in identical generated instances(e.g.,MGAN).Our study focuses on three parts:how to generate small data with high quality,how to address the problem of gradients vanishing and that of mode collapse.(1)This study proposed a new norm penalty to force the generator's loss function to generate simulation data with anomaly,and integrated the Droupout function into the generator to synthesize sparse data with high dimension.Moreover,aimed to different applications,we proposed different evaluation approaches.First,this study explores how to apply the GAN to the small dataset,especially for the industry and medical domains.The common aspect for the two domains is that collecting enough available data is very hard and expensive.Although both of them belong to the small dataset,they have different characteristics.For industrial samples,the amount of negative samples is very small but that of positive samples is very large.For fabric samples,difference between negative and positive samples is the former one hold a very small outlier(image/outlier)?~1/2000).In our study,we term such a dataset“extremely imbalanced data”.The rate of outlier to image in traditional imbalanced data,we take the DAGM as the example,?~1/23.It is hard to generate such an outlier for regular GAN and its variants.Our study integrates such a score which subtracts the outlier from the same size area cropped from the generated instance into the GAN training to produce generated instance with outlier.In this case,this study applies the Faster-RCNN to assess the generated data quality.Second,cancer dataset can be regarded as a sparse array with high dimension.We take the ovarian cancer as the case.The amount of patients is 372 but the dimension is9850.Note that the cancer dataset is different from other datasets from different domains.The cancer dataset consists of two sets.One is the gene dataset,this is used by physician to diagnose whether a person gets the cancer or not;another one is the corresponding clinical dataset of this genome dataset,it mainly consists of the survival status and survival time of each patient.In the former dataset,each column corresponds to one genome.If we observe a certain genome having a mutation,the corresponding value turns out to be 1;otherwise the corresponding value keeps 0.Since mutation is occasionally happened in daily-life,the data density is 0.6%.In this study,we explore how to generate sparse matrix with GAN,and propose a evaluation model to assess the generated data quality with with network-based stratification and Kaplan-Meier algorithm.(2)Aimed at the gradients vanishing problem,our study constructs a special randomized space which contains the original data distribution.The GAN model draws sample from the new space and synthesize new instances.This study explores how to address the problem of mode collapse for GAN.In general,the original data always hold a set of disconnected manifolds.However,the GAN model can usually learn several or even one single manifold.This is because the loss function within GAN just only guarantee that the model can transform the noise into the real data.As to the generated data quality,it is not the major concern of GAN and there is no strategy to enforce GAN generating high quality simulation data.In this way,this study proposes multi-generator mutual information GAN(MIM-GAN)to learn all manifolds of original data.Specifically,a set of generators are employed to learn the disconnected manifolds,and utilizes mutual information to enforce that generators learn different manifolds.This is because the generators may learn the same manifold,resulting in the identical generated instances.A new minimax formula is developed to simultaneously train all components in a similar spirit to regular GAN.Moreover,this study adopts the Maximum Mean Discrepancy(MMD)to assess the similarity score between the original data and the generated data.This is because the Inception Score may be the result of mode collapse.Considering an extreme scenario,all the generated data produced by GAN model are identical.The Inception Score would also be very high.We proceed to conduct extensive experiments,utilizing MNIST(1×28×28),CIFAR10(3×32×32)and CelebA(3×128×128)datasets to demonstrate the significant performance of MIM-GAN and producing diverse generated data at different resolutions.(3)Aimed at the problem of mode collapse,our study employs multi-generator and adopts mutual information to enforce those generators to learn different manifolds,generating diverse simulation data.This study explores how to address the problem of gradients vanishing problem.In the process of training GAN,the generator can be trained to build a mapping function which transforms the noise from the randomized space to real data space.For measuring the dissimilarity between generated data and original data,GAN utilizes the Jensen-Shannon divergence(JS divergence)to measure such a dissimilarity.The smaller the JS divergence value is,the more similar to original data the generated data is.However,if the original data distribution has a negligible overlapping area with generated data distribution in the mapping space,the JS divergence is a constance,and it cannot royally reflect the similarity between the two distributions.To address this proble,our study develops a special noise distribution in which the original data features are contained.Because the noise distribution is contained in the generated data distribution,the overlapping area between the two distributions can be guaranteed during training.In this way,the gradients vanishing problem has been addressed.Specifically,this study adopts Non-negative matrix factorization(NMF)to factorize the original data X,getting the basis matrix W.In addition,the noise is sampled from the Uniform distribution with(0,1).After that,we combine the basis matrix W and the noise to form a new input.The new input would be fed into the generator to be transformed to realistic-like data.The extensive experiments validate the effectiveness of proposed approach.Last,this study discusses the advantages and disadvantages of proposed approach.