Online Handwritten Mathematical Expression Recognition Based On Global Approach

In recent years,the string language,such as LATEX,as well as the template editor is more and more used to input a mathematical expression to science document,however,this kind of input mode for the user's requirements is higher,and needs to be specially trained.With the popularity of touch-screen devices,handwriting input as the most natural human input mode,is used by more and more people.That leads to the birth of handwritten mathematical expression recognition technology.Different from traditional handwriting recognitio n,the mathematical expression itself has two-dimensional structure,in carries on the recognition,we should not only identify the expression of mathematical symbols,but also identify structural relationships between the symbols of the expression.And,symbol recognition is affected by the structure of mathematical expression,and the structure of mathematical expression also relies on the identification symbols.All these make it especially difficult to recognize handwritten mathematical expression.In order to solve this complex problem,mathematical expression recognition is usually divided into three steps,character segmentation and character recognition and the analysis of the structure.Classically,previous steps are executed sequentially.As a result,an error occurring in one step would be inherited by subsequent steps,so that the precision of the final recognition result is not high.Aiming at this problem,this thesis realizes an online handwritten mathematical expression recognition system based on global approach.By using a two-dimensional stochastic context-free grammar,the system considers handwritten mathematical expression recognition as a problem of constructing the largest possible tree,so as to determine the maximum possible recognition result from the global.In this way,the three phases of the mathematical expression(symbol recognition,symbol segmentation,and structural analysis)can be optimized simultaneously.In addition,in terms of symbol segmentation,this thesis proposes to reduce the number of segmentation hypotheses by using the concept of "distance" between strokes.In the aspect of symbol recognition,this thesis proposes a combination of convolution neural network and recurrent neural network to realize the recognition of symbols and improve the accuracy of symbol recognition.In the end,this thesis realizes the function of symbol segmentation module,symbol recognition module and structure analysis through programming,so as to realize a complete online handwritten mathematical expression recognition system.Based on the evaluation results of the system performance and the results of the comparative experiments,the system in this thesis has achieved good results in the expression recognition.On CROHME 2014 test set,the expression recognition rate is 35.18%,ranked third.That proves the system performance is better.Finally,this thesis realizes a handwritten tablet by coding,which can convert the user's handwritten mathematical expression into LATEX format and finally display it.