Research Of Irregular Scene Text Recognition Based On Attention Mechanism

Character is the symbol for recording human thoughts and the tool for exchanging information.The importance of words to human daily life is self-evident.Scene text refers to the text in the natural scene image.The road sign image,billboard image and license plate image all carry the scene text.Recognition of characters in natural scene images has a wide range of applications,such as license plate recognition,card recognition and automatic driving.Several decades ago,the research of text recognition mainly focused on scanned document text.After long-term research accumulation,the method of scanned document text recognition is becoming more and more mature.However,scene text still faces many challenges due to its complex background,variable text presentation and low image quality.Among them,irregular scene text is extremely difficult to recognize because of its irregular character arrangement and character rotation.In recent years,attention based methods have provided new ideas for irregular scene text recognition.The method based on attention mechanism can accurately locate the feature of character region,which has the potential to solve the problem of irregular character arrangement.In this paper,based on the attention mechanism,a series of researches are carried out on irregular scene text recognition:(1).This paper proposes a method for processing irregular text based on attention mechanism and integrating Gabor convolutional neural network.Gabor convolutional neural network can extract more robust features against orientation changes by integrating multi-directional Gabor filters into the convolutional neural network.Sequence recognition network is an attention-based encoder-decoder model,which uses the output features of Gabor convolutional neural network to predict the character sequence.The accuracy of the proposed model was evaluated on multiple scene text data sets,including regular and irregular text.A large number of experiments show that the proposed method achieves optimal recognition performance on multiple scene text data sets.(2).This paper presents a method to optimize alignment performance of attention mechanisms.This method uses the annotation information of the position of the center point of the character and introduces the alignment loss function to optimize the performance of attention alignment.Based on the prior information that the distribution of attention coefficient has a single peak and the peak value is close to 1,the cross entropy loss function is designed for optimization.Based on the prior information of normal distribution of attention coefficient,the loss function of earth-moving-distance is designed for optimization.Both of these loss functions significantly improve the recognition performance?(3).This paper proposes a deformation and multilayer license plate recognition method based on 2D attention mechanism.The method can be used to deal with the plates of multi-layer plates and perspective deformation.In this paper,the YOLOv3 algorithm is modified to improve the performance of license plate detection.The perspective rectification network is used to rectify the license plate,and the 2D attention mechanism is adopted to recognize the single-layer and multi-layer license plates.In order to improve the speed of inference on CPU devices,OpenVINO framework is used as the inference engine and C++ code is used to realize a license plate recognition software that can run in real time on CPU.