Research And Implementation Of Mathematical Expression Handwriting Recognition Technology Based On LSTM Model

During recent years,information technologies,such as Mobile Internet and Cloud Computing have permeated into every aspects of social life.Huge amounts of data are centralized and processed on cloud servers everyday,while the auto-processing and data mining provide us with tremendous economic benefits,new challenges are brought in due to the diversity and complexity of data in real life.Online handwritten mathematical expression data is a kind of data created by people who use terminal devices like touch screens and electronic pens.The handwritings of users are collected in the form of sampling points,and the Online Handwritten Mathematical Expression Recognition technology provides a way to transform the sampling points into text formats,so that the easier processed data formats can be utilized further more.The main difference between Handwritten Mathematical Expression Recognition technology and normal Handwriting Recognition technology lays in the special layout of the mathematical expressions which extend in two dimensions,thus it takes much more effort to recognize handwritten mathematical expressions correctly and this technology is still an open research filed.This thesis studied the key technologies of Online Handwritten Mathematical Expression Recognition,focused on the analysis of the mathematical handwritten formula in the background of real scene elementary mathematics education.We proposed a handwritten mathematical expression recognition solution based on Recurrent Neural Network(RNN)Language Model,meanwhile a handwritten mathematical expression recognition system was implemented according to our studies,and the system was tested on dataset collected from real scene.The major contributions of this thesis are summarized as follows:(1).In the elementary mathematics education field,there are many difficulties in recognizing handwritten mathematical expressions in real scene,for example,the mixture of mathematical symbols and descriptive Chinese characters,and the inherent ambiguity of mathematical symbols.In order to solve these problems,we introduce the usage of language model in sorting recognition candidates,and we also summarized some techniques for the preprocessing of corpus.Our experiments showed that the language model significantly improve the quality of recognition candidates,the top-5 matching rate of recognition candidates increase nearly 20%.(2).We studied the Convolutional Neural Network that has achieved breakthrough results in the field of image recognition in recent years,and used this kind of network to develop symbol recognition engine.In addition,according to the application scenario of the topic,the techniques of introducing illegal categories and data sets sampling methods are specifically proposed,and the symbol recognition engine finally developed has achieved a 99% accuracy of Top5 recognition accuracy on the test data set.(3).This paper studied the extraction of related elementary mathematics formulas,designed a system framework flow of the subject,proposes a structural analysis method for elementary mathematics formulas,and completed the design and implementation of related functional modules.(4).In consideration of the characteristics of mathematical expressions in elementary education field,we proposed a method integrating mainstream deep learning methods,natural language processing and formula structure parsing techniques.Our method can deal with on-line handwrtten mathematical expressions,especially in the situation where mathematical symbols are mixed with Chinese descriptive characters.A complete handwritten elementary mathematical expression recognition system was implemented.And we tested our system with real data,the final results showed that our system was able to provided reliable recognition results in real scene,and was time stable.