Arbitrary Hand-drawn Sketch Recognition

In this thesis, we study the problem of hand-drawn sketch recognition. Due tolarge intra-class variations presented in hand-drawn sketches, most of existing workwas limited to a particular domain or limited pre-defned classes. Diferent from ex-isting work, we target at developing a general sketch recognition system, to recognizeany semantically meaningful objects.Toincreasetherecognitioncoverage, aweb-scaleclipartimagecollectionislever-aged as the knowledge base of the recognition system. A query-adaptive shape topicmodelisproposedtomineobjecttopicsandshapetopicsrelatedtothesketch,inwhich,multiple layers of information such as sketch, object, shape, image, and semantic la-bels are modeled in a generative process. Besides sketch recognition, the proposedtopic model can also be used for related applications such as sketch tagging, imagetagging, and sketch-based image search. Moreover, we also study the problem of howto segment a freehand sketch at the object level. By considering the basic principles ofhuman perceptual organization, a real-time solution is presented to automatically seg-ment a user’s sketch during his/her drawing. First, a graph-based sketch segmentationalgorithm based on entropy of retrieval information is proposed to segment sketch intomultiple parts based on the factor of proximity. Then, for detecting semantically mean-ingfulobjects, asemantic-basedapproachisintroducedtosimulatethepastexperiencein the perceptual system. Finally, other important factors learnt from past experience,such as similarity, symmetry, direction, and closure, are also taken into account intothe system.The proposed sketch segmentation framework can handle complex sketches withoverlappedobjects. Experimentalresultsshowtheefectivenessoftheproposedframe-work and algorithms.