License Plate Detection And Vehicle Detection In Vehicle-Borne Image

License plate detection and vehicle detection are important research topics in intelligent transportation systems,which have been widely used in vehicle identity authentication,smart charging,vehicle density statistics,intelligent driving,and traffic control.In vehicle-borne images,license plate detection and vehicle detection are still challenging due to low resolutions,different shooting angles,and distances.At present,many domestic and foreign researchers have carried out a lot of research on license plate detection and vehicle detection in vehicle-borne images.However,they ignore the relationships between the license plate and the vehicle and the structure of the license plate itself.Specifically,when detecting license plates and vehicles simultaneously,the vehicle detection will not be affected.However,the license plate detection will be significantly affected by the vehicle due to the vehicle containing the license plate.Moreover,the proportion of long-distance license plates and vehicles in the pictures is small,which can easily cause the miss detection of small-scale license plates and vehicles.Besides,the location of the shooting equipment and the license plate may constantly change,making it difficult to detect the multi-directional license plate accurately due to perspective transformation.Given the above problems,we study to eliminate the negative impact of vehicles on license plate detection and improve the detection performance of small-scale license plates and vehicles and multi-directional license plates.Finally,we further verify the effectiveness of the proposed license plate detection methods combined with license plate recognition.The main research contents and innovative work are as follows.1)To effectively detect the license plate and the vehicle simultaneously,we propose to use two independent branches to detect the license plate and the vehicle,respectively.This method decouples license plate detection and vehicle detection and eliminates the harmful interference of vehicle to license plate detection.Moreover,we use feature fusion,anchor clustering,and attention mechanism to enhance license plate detection and vehicle detection.The experimental results show that the proposed method achieves a better detection performance and a faster running speed.2)We propose a novel and applicable method for degraded license plate detection via vehicle-plate relation mining.The proposed method can detect the license plate in a coarse-to-fine scheme.First,we propose to estimate the local region around the license plate by using the relationships between the vehicle and the license plate,which can significantly reduce the search area and precisely detect small-scale license plates.Second,we present to robustly detect the multi-directional license plate by regressing the four vertices of the license plate in the local region.The whole network can be constructed in an end-to-end manner.3)We propose an accurate and fast multi-directional license plate detection approach by readjusting the predicted four vertices of the license plate.First,we present to readjust the coordinate of the predicted four vertices via aligning the minimal rectangle formed by them and the ground-truth box.This way,our method gets a more accurate detection result by alleviating the shift of the vertices.Second,we found bounding box regression and vertex regression are conflicted with each other by experiments.Hence,we remove the bounding box regression module to achieve better performance and faster inference speed.Extensive experiments verify the effectiveness and generalization capability of the proposed method.4)We verify the effectiveness of the proposed license plate detection methods combined with license plate recognition.The multi-branch detection method directly detects the license plate in the whole image,making it easy to lead to miss detection of small-scale license plates.Moreover,the license plate detection method based on the relationships between the license plate and the vehicle will inevitably lead to miss detection of the license plate in the case of the vehicle's miss detection.To solve this problem,we propose a novel license plate detection method that combines the multi-branch detection method and the method using relationships between the license plate and the vehicle.Both methods can predict the four vertices of the license plate and accurately detect multi-directional license plates by using the vertex relationships.Specifically,the proposed method combines the above methods 1).2),and 3)to form an end-to-end license plate detection network.After the vertices of the license plate are detected,the horizontal license plate is obtained by an affine transformation.Then the license plate number is obtained with a sequence-based license plate recognition method.The proposed method can further improve the detection performance of small-scale and multi-directional license plates and enhance the effectiveness of license plate recognition.