Study On The Recognition Of Online Handwritten Flowchart Based On The Grammar

With the rapid development of computer technology, human-computer interaction research began by the traditional human-computer interaction to intelligent human-computer interaction transformation. At the same time, intelligent human-computer interaction has become a research hotspot recently. Sketches as a branch of intelligent human-computer interaction, conform to the people for a long time writing habit, easy to express personal intention, and easy to express personal design idea. In order to let computer usderstand the intention of the sketches, sketches recognition systems and technology become extremely important. Online hand-drawn sketch recognition is an interdisciplinary technology, including computer graphics, pattern recognition, CAD technology and human-computer interaction and other subjects. In this paper, we study online hand-drawn flowchart symbols recognition technology. The online hand-drawn flowchart symbols recognition is divided into three stages:stroke group, symbol classification and syntax parsing of flowchart.In the first stage, stroke group is to gernate stroke combinations which are time and spatial adjacent. We make full use of the temporal and spatial relations between strokes and the dynamic programming algorithm to achieve the stroke group working. We call the stroke combination as hypothesis in this paper.In the second stage of symbol classification, we first train the flowchart symbol as our classifier, and then the classifier is used to classify the stroke combinations for the overall flowchart recognition. In this paper, we use three classifier algorithms, which are the random forest, neural network and the convolutional neural network, to train flowchart symbols recognition, assessed on the classifier’s accuracy, sensitivity, specificity and ROC curve. Finally, we choose the convolutional neural network as our flowchart overall recognition classifier algorithm. We call the output results of this stage as candidate symbol sets.In the third stage, the aim of syntax parsing of flowchart is to obtain the final flowchart recognition result from the candidate symbol sets based on the flowchart of the context structure grammar. Firstly, we recursively definite the flowchart structure grammar according to the structural constraints of the flowchart. Then, we introduce a description language to reliaze the flowchart structure grammar. Finally, we obtain the final recognition result of flowchart recognition from the syntax parsing. We have to do the syntax parsing mainly because of the candidate symbol sets we get from the second stages more than 20 tiems of the flowchart real symbols. Therefore, it is necessary to use of flowchart of context information to obtain the final recognition result.Through the proposed method of this paper, we obtain 80.2% accuracy on the flowchart symbol recognition from the FC Database, a handwritten-flowchart database. At the same time, the recognition method based on the structure grammar can be extended to the other hand-drawn sketch recognition, such as UML diagram, mathematical formula, chemical symbol and so on.