Research On Sketch Segmentation Based On Deep Learning

Human free-hand draxwn sketch has been an important tool for human communication since ancient times.In recent years,with the popularity of touch screen devices such as tablet computers and smart phones,sketch has become one of the most commonly used human-computer interaction methods by virtue of its simplicity,convenience and rich information.However,due to the highly ambiguous,abstract and diverse nature of sketch,understanding sketch for machine is quite challenging.At present,deep learning methods have been widely used in visual images,and the research on sketches has evolved from traditional methods based on contours.regions.skeletons,etc.to deep learning,and fur-ther extends the overall concept of sketches(segment segmentation,sketch retrieval)to a more elaborate concept of semantic segmentation,which will facilitate the development of many sketch-based applications,including sketch recognition and sketch-based image retrieval.In this thesis,we use the deep learning method to transform the traditional sketch segmentation into a Sequence-to-Sequence problem by establishing a deep learning model based on RNN,and the stroke ordering characteristic often overlooked for sketch interpretation is also considered to prormote the stroke-level sketch semantic segmentation process.In addition,based on the existing vector-sketch dataset,this thesis proposes a number of more than 100,000 sketch semantic segmentation dataset by manual annotation and data augmentation to support the training of the model.The main contributions of this thesis are as following.Firstly,a Sequence-to-Sequence model based on Variational Auto-Encoder is proposed,and the result of semantic segmentation is obtained fr-om vector-sketch by encoding-decoding.On this basis,the original model is extended to a multi-category sketch semantic segmentation model by introducing CNN as encoder to obtain the visual features of the sketch.Since the deep learning model requires the support of massive data,this thesis proposes a feasible data augmentation method based on the Google Quick.Draw dataset to construct a large-scale sketch semantic segmentation dataset.Based on this dataset,the algorithms proposed in the thesis are evaluated.We also compare our algorithm with the existing algorithms to embody the feasibility and superiority of the algorithm.Finally,this thesis also evaluates the model on a more complex called SPG dataset and has received good results,further proving the universality of the proposed algorithm.