Research On License Plate Recognition In Unconstrained Scenes

As an effective means to automatically obtain vehicle identity from image,license plate recognition(LPR)has a very wide range of applications in the field of intelligent transportation.In the early stage of development,research on LPR mainly focused on constrained scenes,such as high-speed toll stations,and parking lot entrances and exits.With the popularization of high-definition camera and the increasing demand for social intelligence,LPR in unconstrained scenes,such as electronic police and Skynet monitoring,has gradually become a research hotspot.However,due to the diversity of background environment,lighting condition,and vehicle position relative to camera,not only the captured vehicle image contains complex and varied texture backgrounds such as railings,buildings and billboards,but also the license plate has large translation variation,scale variation,rotation variation and brightness variation,and processing with the existing methods can lead to low recognition accuracy and poor adaptability to scene changes.In unconstrained scene,it is still challenging to study fast,accurate and robust LPR methods.LPR generally includes three parts: license plate detection(LPD),character segmentation(CS)and character recognition(CR).In view of the maturity of character recognition,researches in this thesis are focused on LPD and CS in unconstrained scenes.The major work is summarized as follows.Firstly,aiming at the low accuracy and low efficiency of candidate region extraction in traditional LPD methods,a detection method based on edge density and nearest neighbor clustering is proposed.This method extracts candidate region by using the single-pixel width connected region in the edge image of vehicle as clustering data point,the point with the largest local density as cluster center,and the idea that each point should be assigned to the same cluster as its nearest neighbor of higher density for clutering.In order to improve the clustering accuracy of character edges,a density calculation method based on adaptive window and feature consistency enhancement is proposed.Experimental results on the public LP dataset and Caltech Cars 1999 dataset,and the Traffic dataset composed of road traffic images indicate that the proposed method is more accurate and faster.Secondly,aiming at the problems faced by anchor box based deep learning method in dealing with the scale variation and rotation variation of license plate,a detection method based on neighborhood context semantic deep network is proposed.Initially,the license plate candidate region is obtained from pixel level by a fully convolutional semantic segmentation network.An enhanced segmentation loss function is designed by using the neighborhood context of object,which effectively improves the performance of candidate region extraction.Then,the orientation of object is considered in the region classification and regression network used for verification and location,and the inclined license plate is located accurately.Results of experiments show that the proposed method can effectively improve the detection accuracy of license plate with scale variation and rotation variation,and the detection speed is faster than Faster R-CNN.Thirdly,to improve the segmentation accuracy of character on license plate with frame sticking,tilt distortion and quality degradation,a character segmention method based on multi-scale harrow-shaped template matching is proposed.After the binary image of license plate is obtained through extremal region detection,the method first improves the matching precision of characters by modifying the shape and scale transformation dimension of the traditional matching template,and then reduces mismatches effectively by a cascade filter composed of the local minimum cost criterion,the minimum cost criterion within group,and the maximum correlation coefficient criterion.The experimental results show that the proposed method is robust to degraded license plate images.Finally,by combining with vehicle detection,license plate color recognition and character recognition,a LPR software system is built for overall performance evaluation.Four groups of experiments show that the system is not only robust to license plate scale variation,rotation variation and brightness variation,but also achieves 95.12% recognition accuracy on the LP dataset.On the Traffic dataset,a recognition accuracy of 94.76% and 92.08% is achieved for daytime and nighttime respectively,and the average recognition time is 238 milliseconds,consistant with the demand of the Ministry of Public Security for LPR.