Image-to-sketch Cross-domain Synthesis And Applications In Image Retrieval

Sketch has been used as a simple and effective information medium since ancient times.Even in today's daily life,it is common to use sketch to express one's idea intuitively.With the popularity of touchable smart devices,such phenomenon becomes more and more widespread.As an important information medium,due to sketch's inherent diversity,randomness,and abstractness,sketch synthesis is quite challenging for machine though it is quite simple for human.In addition,the collection of sketch dataset is difficult and costs high labor,which makes related sketch datasets especially image-sketch paired datasets are rare and always possess little data.This situation limits the development of research on sketch.This thesis dedicates to develop an unsupervised deep learning algorithm,which can synthesis stroke-level sketch from real image.On the one hand,the unsupervised training makes paired examples not necessary that the insufficient of data will no more constrains the ability of model.On the other hand,synthesized stroke-level sketch maintains the information contained in stroke sequence,which is quite meaningful for the extension of related datasets.The implementation of image-to-sketch synthesis algorithm needs machine to mimic human's comprehension mode to accomplish a cross-domain task,thus the research in this thesis has heuristic meaning for researches in sketch recognition,sketch semantic segmentation,as well as other cross-domain synthesis algorithm.The main work of this thesis can be summarized as follows.First,an unsupervised image-to-sketch synthesis model has been proposed.This thesis creatively treats image and sketch as two styles of expression and borrows the basic idea in image-to-image translation to solve the image-to-sketch synthesis task.General model structure has been improved to address issues raised by large domain gap between image and sketch.Second,the conversion from spatial information in real image to time-series information in stroke-level sketch has been realized.This thesis analyzes and solves the problem of mismatch between input image and output sketch due to cross-modality.Third,further improvements have been made to make the model compatible with single-category and multi-categories sketch generation tasks.Finally,a visual sketch-based image retrieval application has been realized by applying the synthesis model to realistic image retrieval task.