Online Handwritten Math Expression Label Recognition Based On Long Short Term Memory Recurrent Neural Network

Handwriting is a basic skill for human social communication. With the gradual popularity of mobile terminals, handwriting recognition in real life has become more and more important. Handwritten character recognition has been carried out for a long time. Although the technology is very mature, it is still limited to 1-D space sequence recognition. The recognition technique is not suitable to process 2-D dataset such as handwritten mathematical expressions. Mathematical expression includes complex and diverse information which is not only kinds of symbols, but also space structures among different symbols. In this paper, we use implicit segmentation model to recognize symbols of on-line handwritten mathematical expressions. With this model, we do not need to segment mathematical expression sequences into independent symbols as precisely as explicit segmentation models.The main work and contributions of this paper are:1. Designed the character recognition system of on-line handwritten mathematical expression based on Bidirectional Long Short Term Memory(BLSTM). The system does not need to segment input data into symbol-level. Then use the Connectionist Temporal Classification(CTC) to label the output unsegment mathematical sequences.2. Implemented pre-process and feature extraction of input data of on-line handwritten mathematical expression. The pre-process includes resampling and normalization. The normalization of unsegment mathematical expression sequence is based on the number of strokes.3. Proposed the method of holistic recognition for on-line handwritten mathematical expression symbol recognition. Transfer complex symbols which is made up by several characters into one-char symbols in based on ASCII code in training part. It reduced the uncertainty in tag recognition, and unified the type and size of labels.4. Improved the calculate method of existed label error rate and normalized label error rate which are based on the edit distance to make the evaluation criteria more close to facts.This paper designed one kind of system for on-line handwritten mathematical expression label recognition which can get good recognition results. This system do not need to segment expressions into symbols in pre-process, which can reduce the workload in label recognition. The experiments also proved the availability of holistic recognition method for the complex mathematical symbols, which has higher accuracy than independent recognition method.