Visual Vanishing Point Detection And Its Application To Lane Recognition

Lane recognition can assist the vehicle automatic driving system to determine the position of the vehicle when it is running on the road,and can also judge whether the vehicle has deviated,providing an important reference for the lane departure warning system.Therefore,improving the accuracy and real-time performance of lane recognition is of great significance to the practical process of vehicle automatic driving system and lane departure warning system.Vanishing point(VP)is the intersection of two parallel lines at a very far distance.In outdoor scenes,there are usually multiple VPs.Among them,the VP that is associated with the major geometric structures of the scene and conveys a strong sense of three-dimensional space or depth information to the observer is called the dominant VP.When the traditional method is applied to the dominant VP detection in a complex outdoor scene,the accuracy may be reduced by the interference of the line segment information pointing to different directions.To solve this problem,this dissertation studies the dominant VP detection methods based on deep learning in complex outdoor scenes.On this basis,it explores the road VP detection method and the VP-assisted lane recognition method,and integrates the VP detection and lane recognition together to improve the recognition of lanes in complex scenes.The main research work is summarized as follows:(1)A dominant VP detection network D-VPnet in outdoor scenes based on singleshot object detection framework is proposed.The D-VPnet uses the feature linesegment proposal unit to extract two main parallel lines related to the dominant VP,and then combines the information of main parallel lines with the dominant VP estimation,which improves the detection accuracy of the dominant VP.In addition,Mobile Net V2 is used as the backbone of D-VPnet for improving the detection speed.In order to train the D-VPnet and objectively evaluate the detection performance,a PLVP image data set is constructed.Finally,experiments on the PLVP data set and Zhou’s public data set show that the detection accuracy of D-VPnet is 6.1-22.2% higher than that of Zhou’s method,which has the best performance among the traditional methods.(2)A dominant VP detection network HR-VPnet based on heatmap regression is proposed.This network transforms the dominant VP detection problem into a heatmap regression problem,and combines super-resolution technology and multi-scale supervised learning.The specific implementation uses super-resolution up-sampling to generate a 1/2-scale heatmap,and combines the generated 1/4-scale heatmap to perform multi-scale supervised learning.Experiments on Zhou’s data set and the PLVP data set show that the accuracy of HR-VPnet is higher than that of the D-VPnet network,especially in the range of small consistency error(1～3 pixel).The lightweight reconstruction of the HRNet-W48-M as the backbone improves the real-time performance of the HR-VPnet network.(3)The VP information is effectively introduced into the lane recognition network based on deep learning.Through the discussion of the four proposed fusion modes of lane recognition and VP detection,a multi-task fusion VP-assisted lane recognition network Lane-VPnet is proposed.Experiments on the night road subset,the shadow road subset and the crowded road subset of the CULane data set show that,the LaneVPnet network with VP assistance has better performance in integrity,recognition and anti-interference than the network without VP assistance,and the Lane-VPnet has a good real-time performance,which can reach 96 FPS on NVIDIA 2080 Ti GPU.The research results of this dissertation provide a reference for improving the detection effect of lanes in complex scenes and promoting its application in autonomous driving technology.