Research On End-to-End License Plate Recognition In Complex Scenes

License plate recognition is an important research aspect in the field of computer vision.Many license plate recognition methods and systems have been widely used in parking lots,highway toll stations,and other scenes.The cameras in most existing methods and systems are mainly deployed at fixed angles and positions,leading to high similarity among the license plate images.However,in unconstrained scenes such as safe driving,mobile police,and road safety,camera positions are not fixed.As a result,license plates from these cameras have complex backgrounds and various sizes,with different degrees of shearing and rotation,resulting in recognition errors.Besides,with economic development and social progress,many special types of license plates appear,such as new-energy plate,military-police plate,double-line plate,etc,bringing great difficulties to license plate recognition methods,due to the varying character lengths,unfixed Chinese character position,multi-line layout,etc.Therefore,research on license plate recognition faces a lot of new challenges.To solve the above problems,this paper deeply researches the end-to-end license plate recognition method in complex scenes,and the specific work of this paper is as follows:1.For the problems that the existing methods cannot adapt to the license plate images with changing perspectives and is only able to handle simple scenes,an end-toend single-line license plate recognition network for complex scenes is proposed,achieving license plate detection and recognition in the same network.Firstly,shared convolution is applied to achieve feature extraction from the input images.And feature maps are then entered into the detection branch and recognition branch,respectively.Next,a rectification module is further utilized to connect the above branches.Based on pixel-level segmentation,the detection branch is able to accurately detect the deformed license plates from the images of complex scenes.And with the help of Bi-directional Long Short-Term Memory(Bi LSTM)network,the recognition branch can deal with unfixed Chinese characters position and license plates with varying lengths.The rectification module is adapted to align the features of the license plate area so that the gradient can flow through the entire network during back propagation,achieving end-to-end training and inference.Detection branch and recognition branch work complementary with the shared convolution,efficiently improving the performance and shortening the running time.Experiments on the AOLP dataset show that the accuracy of the proposed method is 0.86% higher than the best method.On the four public datasets such as SSIG,the proposed method achieves state-of-the-art both on speed and accuracy.2.For the problem that existing methods cannot recognize the license plate with multi-line layout,this paper expands the single-line license plate recognition network and proposes a feature recombination based end-to-end multi-line license plate recognition network.Firstly,a shallow classification module is embedded in the network to distinguish the layout type of license plates.Then,feature recombination is applied with cutting,pooling,and joining operations to recombine features from different lines.Finally,Bi LSTM is used for recognizing recombined multi-line features.The proposed method efficiently simplifies the recognition process of the multi-line license plates without extra line segmentation or character segmentation operations.In addition,synthetic license plate images are used for training to solve the problem of insufficient training data.Experiments show that the proposed multi-line license plate recognition network improves the accuracy by 7.85% compared with the segmentation based method and achieves the accuracy of 98.29% on the benchmark dataset.Besides,this paper optimizes the end-to-end network,with 130% improvement on the inference speed than before the optimization.