The Research On Attention-based Chinese Text Recognition

As one of the greatest inventions of mankind,text is not only a written expression of human language,but also a spiritual and cultural heritage.While image,as an important information carrier,not only carries information such as color,texture and structure,but also always has large number of texts.However,the text often contains more abundant semantic information,which may provide important clues for understanding the image scene.It can be said that texts can be seen everywhere in daily life.How to identify and understand text in images has important research significance,and also has a wide range of practical applications.Applications such as handwritten note recognition,license plate recognition and photo translation can greatly improve human productivity and bring convenience to people's lives.Although the traditional document text recognition technology is near perfect,accurate recognition of handwritten Chinese characters and text in natural scene pictures is still a very challenging task due to the huge difference in handwriting style and the complex background of natural scenes.With the rapid development of deep learning,the computer vision field has ushered in a major breakthrough.Based on the deep learning technology and the characteristics of Chinese,in this thesis,we explore the recognition of handwritten Chinese characters and natural scene chinese text.The main contents of this thesis are as follows:(1)A convolutional neural network with multiple comparative attention is proposed in order to produce multiple local attention regions.Supervised by multi-loss,the learnt attention feature can be focused on discriminative patterns across different categories and the attention map from the same class will be located at the same regions to further obtain robust features invariant to large intra-class variance.Compared with the previous methods,the recognition error rate of both inter-class and intra-class characters can be effectively reduced with our method.As a result,the proposed method outperforms all single-network methods trained only on handwritten data.(2)A novel dual attention network for Chinese scene text recognition which combines1-D and 2-D attention mechanism is proposed.Most previous 1-D sequence based method can not handle the irregular text such as multi-orientated and curved text.To address this limitation,a scene text recognition method based on encoder-decoder framework is presented.The 2-D attention mechanism is applied on the feature map which selects local features with rich spatial information for individual characters in a weakly-supervised way.Then,the 1-D attention mechanism is used to obtain richer semantic features of character from the output feature sequence of encoder.Finally,two kinds of attention features are adaptively integrated to do classification.Different from previous methods,the proposed model dose not rely on sophisticated designs(including spatial transformation,character instance segmentation or multi-directional encoding)and can be trained end-to-end without any additional pixel-level or char-level annotations.Benefiting from two kinds of attention mechanism,the proposed method is robust and can achieve great performance on both regular and irregular scene text recognition.