A Hybird Framework For Physics Problems Recognition

Physical problem recognition aims to automatically read physical problem from the image,which can support many intelligent education applications such as machine solution,auto-matic correction,automatic search,etc.,and has important application value.This paper mainly studies the automatic recognition of physics test questions in middle school.In mid-dle school physics problems,the stem of questions contains multimodal data such as words and figures,while the texts include Chinese characters,numbers,English letters,Greek let-ters(physical units),etc.The structures of different modes are different,so it is difficult to put forward a unified method to read them.Based on the above analysis,combined with the advantages of deep learning and traditional OCR technology,this thesis proposes a multi-stage solution for the task of middle school physics problems recognition.Specifically,first of all,image segmentation is carried out,and the bottom-up method is used in the process of image segmentation.Then,the traditional algorithm is improved,and the method of projec-tion and connected domain are used to improve the segmentation accuracy.Then,the neural network is used for classification operation,combining the traditional algorithm with the excellent deep learning model to improve the recognition accuracy.In the process of classi-fication and identification,first of all,Chinese and non-Chinese binary classifications need to be carried out,the binary classifications are experimented using ResNet,DenseNet and Mo-bileNet respectively,and the experimental results are compared;the non-Chinese(symbols)obtained from the binary classifications need to be multi-classified,the multi-classification is experimented using ResNet and InceptionV3 respectively,and classification results are obtained and compared.After optimization,the accuracy of character segmentation reaches 93.36%.The data set after the segmentation of physical test questions is collected and dis-closed,including 11274 samples,which can promote the progress of the research field.The recognition rate of the binary-classification and multi-classification model is over 90%.And finally,identify the accuracy of physics problems by the overall framework reach 86.88%.