Research And Implementation Of Mathematical Formula Recognition System Based On Deep Learning

In recent years,with the rapid development of computer technology and the popularity of mobile phone shooting function,people often start to use mobile phones to take photos or screenshots to store information.For text information in pictures,Optical Character Recognition（OCR）technology can be used to recognize text information in images and convert it into editable computer documents.For traditional text information,OCR technology has been very mature,with a very high recognition rate.As for the mathematical symbols and formulas in the pictures,the traditional OCR technology can not deal with the complex two-dimensional structure of mathematical information well,so the recognition effect is not satisfactory.Especially for pictures taken through mobile phones,many formulas are completely unrecognizable due to the influence of illumination,camera angles and blurring.Therefore,it is of great practical significance to design and implement a system to detect and recognize mathematical information in pictures taken by mobile phones.The traditional formula recognition technology uses character segmenta-tion and grammar reconstruction,and the recognition accuracy is often lower than 70%.With the rapid development of computer vision and deep learning theory and algorithm,the recognition accuracy of mathematical formula detec-tion and recognition system based on these two technologies has been greatly improved.These systems mainly use the images obtained by the scanner as training data,and the recognition accuracy rate can often reach more than85%.However,experiments show that their recognition accuracy is usually lower than 80%when processing pictures taken by mobile phones.This paper studies the detection and recognition of mathematical infor-mation in pictures taken by mobile phones.The specific research content is as follows:1.The fuzzy kernel was estimated by simulating the fuzzy kernel path,and the fuzzy image was generated.A data set for motion blur removal of text image was constructed,based on which a model suitable for text image blur removal was trained by DeblurGAN-v2 algorithm.An improved MBCNN algorithm is proposed to remove the moire lines produced when shooting digital screens.2.For the problem of low recall rate of small target detection by the YoLo-FastestV2 algorithm and variable size of mathematical formulas,this paper uses an improved FastestV2 algorithm for mathematical formula test-ing.The improved model is fast enough to complete the monitoring of math-ematical formulas while maintaining real-time performance.Compared to the original model,the F₁ score is 88.2%from 77.8%,and the recall rate is 92.2%from 68.8%.The improved YoLo-FastestV2 algorithm model is an obvious advantage in small target detection.3.Aiming at the problem of low resolution of images intercepted by target detection,this paper adds an image enhancement module to the formula recognition algorithm to enhance image resolution,and expands the formula images intercepted by target detection in the previous step to the formula recognition data set.Based on this data set,the trained formula recognition model has strong generalization performance.4.This paper designs and implements a complete mathematical formula detection and recognition system,which is suitable for pictures or screenshots taken by mobile phones.At the same time,it designs a UI interface which is convenient to operate.The images of the detected mathematical formulas are automatically captured and input into the formula recognition module.Finally,the LaTeX sequence of each formula is output.