Printed Mathematical Formula Detection And Recognition Based On Deep Learning

With the advent of the information era,information increasingly needs to be transmitted and stored quickly and accurately.And the transformation of paper media into electronic me-dia has become a mainstream.Therefore,detecting and recognizing printed mathematical formulas from a document and outputting them as computer editable formats has become a research hotspot and a tough issue to be solved in the field of computer vision,which embod-ies profound research significance and great application value.This thesis conducts in-depth studies on the detection and recognition of printed mathematical formulas based on deep learning,establishes a diverse and complete mathematical formula database,and improves and optimizes the neural network model according to the particularity of the mathematical formula structure.Moreover,this thesis designs and implements a set of complete mathemat-ical formulas detection and identification of the mobile terminal system.The main content of this thesis is as follows:(1)In this thesis,a large number of images containing mathematical formulas are selected according to the dimensions of language background,formula type,interference level,image resolution,purpose,etc..Four databases are established according to these above-mentioned dimensions while a variety of data enhancement techniques are applied to expand the database.Aiming at different interferences,this thesis adopts the modified binarization algorithm,im-age denoising algorithm and tilt correction algorithm to process to data.As for the removal of interference information,the database retains effective information to the maximum extent,so as to lay a good foundation for subsequent detection and recognition.(2)This thesis chooses Faster R-CNN with relatively better performance from three main-stream target detection algorithms in the task of mathematical formula region detection.In view of the particularity of the structure and characteristics of the mathematical formula,the backbone network,anchor frame specifications and pre-training network of Faster R-CNN were improved to make the chosen Faster R-CNN more suitable for detecting the mathemat-ical formula area in the image,improving the detection accuracy and reducing the operation time.Experiments have shown that the overall detection accuracy of the improved algo-rithm in this thesis is slightly higher than the YOLOv3 algorithm and the unimproved Faster R-CNN algorithm while the improved algorithm can reach 85.60%.(3)In the task of mathematical formula recognition,this thesis uses a codec recognition model based on the attention mechanism to replace the CRNN+CTC model,which features good recognizing one-dimensional text.This thesis studies an attention mechanism from coarse to fine to replace the standard attention mechanism.And this thesis attempts to largely reduce the operation time at the cost of very small recognition accuracy.According to ex-periments,the recognition accuracy of the improved algorithm applied in this thesis is much higher than the CRNN+CTC model,and slightly lower than the standard attention codec model while the improved algorithm can reach 71%.(4)Based on the above algorithm research,this thesis designs and implements a mobile terminal mathematical formula detection and recognition system.The proposed system can greatly reduces the size of the network model through cropping and compression,making the system embrace the portability of the mobile terminal and applied to actual scenarios better.The experimental results have shown that the system better realizes the function of mobile terminal mathematical formula detection and recognition.