| Image generation is a research hotspot in the field of computer vision at present,and image generation based on sketch is one of the important research directions,which has a wide range of application scenarios in the fields of game,film and art creation.Existing sketch-based image generation methods mainly solve the problem of instance sketch image generation,that is,generate images from a single sketch object.The generation of high-quality images from scene sketches is now almost empty.This paper focuses on the image generation algorithm of scene sketch,and explores two generation directions from multistage and end-to-end,enriching the image generation means and methods based on scene sketch.The research contents and innovations of this paper are as follows:(1)A two-stage generation model Multi-Sketchy GAN based on scene sketch is proposed,which firstly uses a generative adversarial network to generate foreground,and finally sends the generated foreground image and background sketch into another generative adversarial network to generate scene level image.In the stage of foreground generation,this paper proposes a new image generation network based on instance sketch,which uses AdaLIN to retain content information and global information,uses attention module to guide the model to focus on the brushstroke intensive areas in the sketch,and uses an auxiliary classifier to encourage the model to generate images that meet the target class.In the background generation stage,based on the deep residual network,the paper uses the position-space attention module and the proportional channel attention module to improve the structure generation correlation of spatial and channel dimensions,and effectively improve the quality of the generated image.Experimental results show that compared with many competitive algorithms,the proposed Multi-Sketchy GAN has improved both subjective effect and objective index.(2)An end-to-end generation model SSGAN based on scene sketch is proposed,which makes up for the complex training and parameter tuning and tedious data processing problems of the two-stage generation method.The model proposes a semantic fusion module based on semantic GauGAN network to solve the problem of sketching generating an image by jointly learning semantic and image features.In the discriminator part,a parallel semantic discriminator is constructed for the first time to encourage the model to learn the correct semantic distribution.Experimental results show that:a.Compared with the two-stage model proposed in this paper,the one-stage model proposed in this paper has better objective index and subjective effect in FID,although there are shortcomings in shape and detail of foreground object generation;b.At the same time,compared with existing algorithms,the subjective effect and objective indicators of the proposed model are improved.(3)Design and implement an image generation system based on sketch.On the basis of the algorithm research,this paper designs and implements an image generation system based on B/S architecture,and verifies the effectiveness of the algorithm.The client of the system has the functions of drawing sketches,uploading sketches and generating images.The server is responsible for accepting the client’s request and calling the corresponding image generation method according to the request and transferring the generated images to the client. |
