Research On Algorithms For Detection And Recognition Of Handwritten Mathematical Expressions

Computer document analysis and understanding technology has a wide range of applications in digitization of paper documents,scene understanding and information retrieval.With the deep integration of information technology and education and teaching,intelligent education has become one of the research hotspots.The automatic analysis and comprehension of educational documents and the automatic marking system are the important contents of intelligent education.Educational documents often contain a large number of complex mathematical expressions.In this paper,its research mainly includes detection and recognition of mathematical expressions.Expression detection is to precisely locate expressions in a document;Expression recognition is to convert mathematical expression images into structural expressions,such as La Te X mathematical expressions,graphs or symbol layout trees.Expression recognition not only needs to recognize all symbols from the image,but also needs to give the two-dimensional spatial position relationship between the symbols;And handwritten expressions have the characteristics of uncertainty in writing methods and diversity of writing styles.Therefore,high-precision handwritten expression detection and recognition is a great challenge.This paper studies the detection and recognition of handwritten mathematical expressions,and proposes an expression detection algorithm based on Efficient Net,an expression recognition algorithm based on codec structure and a recognition algorithm based on graph neural network.The main work of this paper is as follows:1.A detection algorithm based on Efficient Net is proposed for the detection of handwritten mathematical expressions.The algorithm consists of a feature extraction network and a detection network.First,the Efficient Net network is used to extract multiscale features,and then the bidirectional feature pyramid network(Bi-FPN)is used to fuse the multi-scale features.The detection network implements expression detection through pixel classification and regression.In addition,regional shrinkage strategy and data enhancement strategy are adopted to further improve the accuracy of expression detection,which lays the foundation for subsequent expression recognition.2.A codec-based handwritten mathematical expression recognition algorithm is proposed.In order to improve the recognition accuracy,this paper uses the position information embedding and attention mechanism to improve the recognition performance,and jointly trains the detection network and the recognition network to improve the model performance,inference speed,and reduce the number of parameters.3.A handwritten mathematical expression recognition algorithm based on graph neural network is proposed.First,we use the detection algorithm to locate the symbols inside the expression,and then extract the Ro I(Region of Interest)feature for each symbol inside the mathematical expression,and use the graph neural network with the attention mechanism to recognize the LOS(Line-of-Sight)graph.And the graph structure representation of mathematical expressions is obtained,which improves the interpretability of expression recognition.In addition,semi-supervised learning is adopted to make full use of unlabeled data to further improve the performance.The algorithms proposed in this paper achieved 94.1% F-measure in the expression detection experiment on the self-built dataset HAED,and 68.4% Exp Rate in the expression recognition experiment,and improves the F-measure of expression detection through End-to-End training to 96.8%.In the expression recognition experiment on the self-built dataset HAED-Graph,the Exp Rate of 59.33% and the Stru Rate of 67.33% were obtained,the results of ablation experiments demonstrate the effectiveness of the algorithms.