Natural Scene Text Detection And Recognition Based On Deep Learning

Reading text information from image / video is of great value to image recognition / retrieval,geographic location,office automation,helping blind people and other rich practical applications,because scene text contains very useful semantics to understand the world.In recent years,reading text in scene images has become an active field.Scene text reading provides a fast and automatic method to obtain text information in natural scenes,which is usually divided into two sub problems: scene text detection and scene text recognition.Thanks to the strong performance of deep neural network,scene text detection and recognition has made remarkable progress.Based on the scene reading method of deep learning,this paper designs an end-toend scene text reading system,Mask Reader,which is mainly used to detect and recognize English and numbers in images.The main work of this paper is as follows:(1)Based on Mask R-CNN,the problem of text detection is solved by instance segmentation,so that it can detect any shape of text.(2)We demonstrate that using the Pyramid Attention Network(PAN)as a new backbone network of Mask R-CNN enhances the feature representation ability of Mask R-CNN significantly.(3)Mask Reader is the first text framework that can be trained end-to-end.It has a simple and smooth training scheme,so its detection model and recognition model benefit from feature sharing and joint optimization.Different from the previous point sampling method,which only deals with horizontal or directional text,this method can sample any shape of text,including horizontal,directional and curved text.(4)Further,a spatial attention module is proposed to enhance the performance and universality.Benefiting from the proposed two-dimensional representation on both detection and recognition,it easily handles text instances of irregular shapes,for instance,curved text.(5)We verify it on multilingual datasets to prove its robustness.The results show that the most advanced performance has been achieved in both text detection and text detection on these datasets.