Research On Image Modification Method Based On Generative Adversarial Network

After years of development,the generative adversarial networks,which was born in 2014,has become one of the mainstream algorithms in the image field.Text-generated images and image text annotation are hot topics in the field of computer vision and natural language processing,both of which are studying the correspondence between text and images.Since the mutual generation of images and texts can be achieved based on the correspondence between text and images,the images can be modified according to this correspondence.With the rapid development of Internet bandwidth and mobile devices,the need for image modification is getting higher and higher in people's daily work and life.However,common image modification software often has high operating thresholds.Therefore,it is of great significance to carry out in-depth research on image modification methods and propose an image modification method based on generative adversarial network combined with deep learning.In light of the above background,this thesis focuses on image modification methods based on generative adversarial networks,and proposes a new method for modifying image content based on input text information,which is able to make the generated image and the original image similar as a whole but different locally.By analyzing studies on generative adversarial networks at home and abroad and the application of generative adversarial networks in the image field,this thesis proposes a new method for image modification and achieves the following results:(1)Firstly,this thesis proposes a semantic control image modification method based on generative adversarial networks.The Skip-Thoughts model is used to encode the image description to obtain the semantic information,which is added to the generator and the discriminator,so that the generator can generate images according to the information.Then,the image is modified by introducing the reconstruction losses of the generated image and the original image,which makes the two images similar as a whole but different locally,(2)Secondly,this thesis improves the semantic control image modification method based on generative adversarial networks.The fastText model is used to encode image descriptions at word level to obtain more fine-grained semantic information,and the attention mechanism helps to separate the image into foreground and background.The generator and the discriminator only center upon the foreground of the image,and the final result is a fusion of the generated image and the original image.A local discriminator is introduced into the discriminator to judge whether the generated image and the input description match explicitly on multi-scale,so as to improve the discriminating ability of the discriminator and then indirectly improve the generating ability of the generator.Additionally,the use of joint training and phased training can solve the problem of difficult attention model training without a pre-labeled attention map.