Research On Sketch Generation Based On Neural Network

Sketch is a communication tool commonly used by human,which is natural,concise and abstract.It can quickly depict objects,outline figures,and design buildings.The abstract expressiveness and convenience of sketches-enable it to be effectively combined with practical applications such as image retrieval and 3D model reconstruction.The research on sketch understanding and processing has always been one of the hottest topics in the field of computer vision.Affected by personal drawing styles,there are large differences in the abstraction,form and style of sketches,so sketch-oriented semantic understanding is a very challenging task.Sketch generation is an important part of the research on sketch understanding.It can automatically generate sketches based on the computer's understanding of the semantics of sketches.It is not only a manifestation of the results of sketch understanding,but also a driving force for promoting the development of sketch understanding.However,the existing sketch generation models have the problem that the quality of generated sketches is low and cannot support multi-category sketch generation.Therefore,this paper constructs two sketch generation models based on the stroke sequences and lines of sketch respectively,which can generate high-quality sketches based on different input.The main work and innovations of this thesis are as follows:(1)Proposing a sketch generation model based on stroke sequences.By understanding and re-encoding the relationship between the sequence of sketch strokes,more diverse and realistic sketches are generated.This model combines the VAE model with the GAN model,and applies a specified prior distribution to the hidden vector output by the encoder,making the distribution of the hidden vector has a certain randomness and is closer to the true distribution.Experiments on the QuickDraw data set show that the sketch generation model proposed in this paper can generate richer,high-quality sketches,and the classification accuracy of generated sketches is about 5%higher than the previous state-of-the-art sketch generation model.(2)Proposing a sketch simplification model based on lines,which uses the convolutional neural network to extract the visual features of important lines in the original sketch and generate clear simplified sketches.This model introduces the self-attention mechanism and perceptual loss to improve the performance of our sketch simplification model by making full use of the deep semantic features in the original sketch.The results of multiple groups of user researches show that the simplified sketch generated by our model is superior to other models in all aspects such as image aesthetics,line simplicity and content consistency.(3)In order to solve the problem of lack of training data set for sketch simplification model,a new paired rough and clear sketch data set with supervision is constructed in this paper,which provides a good data foundation for the development of sketch simplified research.