Tibetan-chinese Bilingual Natural Scene Text Detection And Recognition System

With the rapid development of intelligent technology,many applications need to utilize rich text information in the scene,such as autonomous driving,traffic monitoring and smart city construction.As for the text in the scene image,the text type is different,the background is complex,the shape is diverse,the scale is variable,and it is easily affected by physical factors,which makes the detection and recognition of the scene text much more difficult than the traditional scanned document image.Therefore,text detection and recognition in natural scene images is a very challenging task and has become an important research branch of artificial intelligence technology.In Tibetan areas of China,Tibetan-Chinese bilingual scenes can be seen everywhere,such as merchant plaques,road signs,newspapers and periodicals,etc.They carry important tasks such as education,publicity and information visualization,and play an important role in people's study and life.Most of the existing research on scene text detection and recognition focuses on Chinese,English and other mainstream languages,and few studies on scene text recognition in Tibetan and Chinese simultaneously.Based on the research of scene text detection and recognition based on deep learning,this graduate thesis designed and implemented a Tibetan-Chinese bilingual scene text detection and recognition system.The main work is as follows:Firstly,view of the present situation that there is no public standard Tibetan and Chinese bilingual scene image dataset,explore methods and specifications for constructing a Tibetan-Chinese bilingual natural scene text detection and recognition database,and use manual annotation and artificial synthesis methods to construct 9548 images for text detection and 71277 images for text recognition,which lays a data foundation for the study of text detection and recognition in Tibetan and Chinese scenes.Secondly,a text detection method based on expansion of text center region for Tibetan-Chinese bilingual is proposed to realize the effective detection of Tibetan and Chinese Texts in the scene.Use Res Net as the backbone network,followed by the feature pyramid enhancement module FPEM to enhance the input feature pyramid;use the FPN network to reconnect the features of all scales for feature fusion;on the combination of feature maps of different scales,predict the rectangle text region,the text center region and the distance from the text center region boundary to the full-text instance boundary,so as to expanded from the text center region to the full-text instance that to complete the text detection of arbitrary direction and shape in the scene image.As result,the proposed method achieves a precision of 75.47% on the real scene text detection dataset of Tibetan-Chinese bilingual.Thirdly,based on the method of convolutional recurrent neural network and connected temporal classifier(CRNN+CTC),the recognition model is obtained by training the scene image of Tibetan and Chinese bilingual text lines.First,the convolutional neural network is used to extract the convolutional features of the input image and encode them into feature sequence,and then the feature sequence is weighted by a fully connected network.Second,a Bi GRU is used to learn each feature sequence,and the probability distribution of the label is predicted through the fully connected layer.Finally,the predicted label sequence is mapped to Tibetan/Chinese words through CTC to obtain the recognition result.The best recognition accuracy is 85.31%,which has good recognition performance.Finally,the scene text detection and recognition system of Tibetan Chinese bilingual is designed and implemented.The system integrates the models obtained by the proposed algorithm and combines the front-end technology,detects the text of the image uploaded by the user,and then recognizes the text region which from detect result into computer-coded of Tibetan and Chinese text,and displays them on the system interface.Through the test of different functions,it shows that the system achieves the expected effect,runs smoothly and has low time complexity,and has strong application value.