High Quality Sketch Generation Based On VAE And GAN

Freehand sketching is a medium for humans to exchange ideas and plays an important role in communication and design.In recent years,the deep generation model has risen rapidly in the field of raster image generation,and the generation of hand-drawn sketches has also received widespread attention.Sketch-pix2 seq is currently the most popular generation model in the field of hand-drawn sketch generation,but it cannot capture the global position relationship of components.This problem is more serious when there are many sketch components;At the same time,the existing sketch generation model is subject to VAE(Variational Auto-encoder)framework,it is easy to generate sketches with unclear details.In order to improve the quality of generated sketches,the two sketch generation models based on different architectures were proposed in this thesis:(1)A hand-drawn sketch generation model based on VAE and attention mechanism.Aiming at the problem that the model Sketch-pix2 seq cannot learn the global position relationship of components,a generation model ESke VAE(Enhanced Sketch Variational Auto-encoder)based on VAE and attention mechanism was designed in this thesis.The model integrates the attention module in the encoder part to comprehensively learn the local structural features and global structural features of the sketch.In addition,for the problem that there is no clear evaluation method for hand-drawn sketches,a subjective evaluation index based on the sketches' own characteristics was designed in this thesis.The experimental results show that the quality of the ESke VAE model is better than SketchRNN and Sketch-pix2 seq,and the quality improvement is more obvious when there are many sketch components.(2)A model of hand-drawn sketches fused with ESke VAE and GAN(Generative Adversarial Networks).Aiming at the problem that the existing sketch generation method is easy to generate sketches with unclear details,a generation method ESke VAE-GAN that combines ESke VAE and GAN was designed in this thesis.The model is based on the VAEGAN generation framework and contains three constituent modules: encoder,generator and discriminator.The encoder and the generator form the VAE structure,and the generator and the discriminator form the GAN structure.The GAN is used to make up for the shortcomings of the VAE architecture.Experimental results show that the ESke VAE-GAN model further improves the model quality based on the ESke VAE model.Compared with Sketch-RNN and Sketch-pix2 seq,the quality of generated sketches is significantly improved.