Sketch Recognition With Natural Correction And Editing

With the increasing popularity of touch-screen devices, there is an increasing interest in building practical system to recognize sketched symbols and diagrams. The goal of this work is to move from traditional point-click-drag style of interface to a more natural interface based on sketch recognition technology, which will make ?owchart/-diagram creation more natural and easier. In this thesis, we study the problem of sketch recognition. We focus on developing robust and e?cient sketch recognition framework that 1) enables both users’ natural interaction and 2) enable various(e.g. online/o?ine)sketch input. Our work has two parts: ‘online sketch recognition’ and ‘o?ine sketch parsing’.In ‘online’ part, we systematically study how to incorporate users’ correction and editing into isolated and full sketch recognition. This is a natural and necessary interaction in real systems such as Visio where very similar shapes exist. First, a novel algorithm is proposed to mine the prior shape knowledge for three editing modes. Second, to differentiate visually similar shapes, a novel symbol recognition algorithm is introduced by leveraging the learnt shape knowledge. Then, a novel editing detection algorithm is proposed to facilitate symbol recognition. Furthermore, both of the symbol recognizer and the editing detector are systematically incorporated into the full sketch recognition. Finally, based on the proposed algorithms, a real-time sketch recognition system, namely ‘SmartVisio’, is built to recognize hand-drawn ?owcharts and diagrams with ?exible interactions. Extensive experiments show the effectiveness of the proposed algorithms.In ‘o?ine’ part, we propose to use shapeness estimation for the problem of o?ine sketch parsing. Our contributions are two-fold. First, motivated by objectness detection, a shapeness estimation method is proposed to perform fast shape/non-shape classification. In shapeness detection, we use a novel 256 D feature which is more suitable for sketched symbols. Experiments show that our shapeness detection method gives much higher recall than competing state-of-the-art object detection methods. Second,based on the proposed shapeness estimation, we present a unified and e?cient threestage cascade recognition framework to parse o?ine/online sketch. Different from previous methods, the shapeness estimation technique in our novel sketch recognition framework e?ciently generates a small set of candidate stroke groups. Therefore, our framework achieves better performance in both recognition accuracy and e?ciency in the public dataset than previous methods. Our framework also has the potential to deal with different types of input sketches, such as online sketch, o?ine sketch or real-word photo.