Sketch Generation And Reconstruction Based On Different Modal Data And Its Application

There is a long history of using sketches as an efficient and simple way to convey information.In everyday communication,it is common to use sketches to convey ideas.With the proliferation of smart devices,the brushstrokes left by people touching the screen are transformed into sequential expressions different from raster diagrams.In the process of sketching,humans are influenced by the state and environment,and the final sketch generated may be different from the one envisioned.In the field of sketch generation and reconstruction,it is difficult to make computers learn and understand the nature of sketching from the process of human sketching.This thesis is devoted to investigate a model of sketch generation and reconstruction based on different modalities for transforming the vectorized storage of human drawing sketches in computers into visual raster maps.On the one hand,there have been many studies on models that only focus on a single data representation or the same modality of input and output.But the data representations are very different.So it is important to study sketch models that can cope with different modalities of data.On the other hand,the abstract features of the sketch itself and the completeness of the image are a set of contradictions.Therefore,the task of how to repair and reconstruct the original image from a potentially fragmented sketch is of great significance.Combining the two,the research in this thesis is heuristic for a cross-modal sketch generation task with both sketch repair and retrieval capabilities for generation algorithms in this area.There are mainly three contributions in this thesis:firstly,this thesis proposes a sketch generation model based on different modal data.This model innovatively transforms the harsh visual raster graph input into a more universal value of point and graph input,and proposes a solution for the model to cope with different modal data.Secondly,a new encoding approach is proposed for the sketching task,which uses the features of key points as an efficient graph model.Finally,the use of graph convolution networks combined with residual networks enables the model to repair the graph representation of sketches.The decoder can complete the missing parts and retain the drawing features of the original sketches.Based on cross-modal data,model makes the repair of defective sketches and the retrieval of defective sketches possible.This makes it possible to repair defective sketches and retrieve defective sketches for such applications.