Research On Roadway Obstacle And Lane Line Detection Based On Computer Vision

With continuous advancements in the development of autonomous vehicles,the perception requirements of these vehicles regarding the surrounding environment are constantly increasing.It is notable that roadway lane line and obstacle detection plays an important role in environmental perception algorithms.The environmental perception algorithms provide essential environmental information for the decision-making and control of autonomous vehicles.The accuracies of traditional machine learning algorithms no longer meet the requirements of autonomous vehicles’ operations.The deep learning algorithms have been developed continuously,and significant progress has been made in roadway lane line detection and obstacles detection.However,it is well-known that the camera’s performance is often affected by night,rain,fog,strong light,and other conditions,which affect the detection and prediction performance.We discuss how the proposed algorithm overcomes these challenges by using deep learning and giving good accuracy.This thesis is divided into three parts.The first part discusses the unknown obstacles detection aims to detect the(drivable)road surface ahead vehicle and plays a crucial role in the driver assistance system.A new obstacle detection method based on Markov random field(MRF)and convolutional neural network(CNN)is proposed to improve the accuracy and robustness of unknown obstacle detection approaches in complex environments.Small and medium-sized obstacles such as rocks,small boulders,and bricks left unattended on the road can pose hazards for autonomous and human driving situations.Often,these obstacles are too low on the road and go unnoticed on depth and point cloud maps obtained from state-of-theart range sensors such as 3D Li DAR.Firstly,I propose a novel algorithm that fuses appearance and 3D cues such as image gradients,curvature potentials,and depth variance into the MRF formulation that segments the scene into the obstacle and non-obstacle regions.Appearance and depth data are obtained from a ZED stereo pair mounted on a Husky robot,and an electric vehicle is used.While accurately identifying true positive obstacles such as rocks and large stones,our algorithm is robust to false-positive sources such as appearance changes on the road,papers,and road markings.High accuracy detection in challenging scenes,such as when the foreground obstacle blends with the background road scene,indicates the proposed formulation’s efficacy.Secondly,the main objective of our work is to achieve autonomous driving with a light deep neural network suitable for deployment on embedded automotive platforms.The network architecture,computational complexity,and performance evaluation during autonomous driving using our network are compared with two other CNN that we reimplemented to evaluate the proposed network objectively.The trained model of the proposed network is four times smaller than the Pilot Net model and about 250 times smaller than the Alex Net model.While the complexity and size of the novel network are reduced compared to other models,which leads to lower latency and higher frame rate during inference,our network maintained the performance,achieving successful autonomous driving with similar efficiency compared to autonomous driving using two other models.Moreover,the proposed deep neural network downsized the need for real-time inference hardware in terms of computational power,cost,and size.The second part discusses real-time lane line detection and lane offset estimation challenges in complex traffic scenes.Traditional lane line detection methods need manual adjustment of parameters.It confronts many issues and challenges,and it’s still vulnerable to interference from obstructing objects,lighting fluctuations,and pavement degradation.It is also challenging for researchers to design a robust lane line detection and lane offset estimation algorithm.Therefore,a convolutional neural network(CNN)model has been proposed for lane line detection and lane offset estimation in a complex road environment,transforming lane line detection problems into an instance segmentation.It also provides a global scale perception optimization approach to address the issue of vanish point lane line width.However,real-time lane line detection and lane offset estimates in complex traffic conditions have proven hard in autonomous driving research.Therefore,the symmetric kernel convolution of classical CNN is upgraded to the structure of asymmetric kernel convolution(AK-CNN)based on lane line detection,and lane offset estimation features.It reduces the CNN network’s computational load and improves the speed of lane line detection and offset estimates.The experiment was carried out on the CULane dataset.A complex environment was detected by lane line detection with high accuracy of 80.3%.The detection speed is 84.5fps,making it possible to track the lane line.The third part mainly discusses developing an intelligent system for the autonomous vehicle where the lane marks are missing or the unstructured road.Currently,lane line detection algorithms are mainly based on visual feature information(such as color,grayscale,and edge),while the environment dramatically affects the detection accuracy.However,the lane line length,width,and direction have strong regularity,serialization,and structural association characteristics which are not easily affected by the surrounding environment,such as visibility,weather,and obstacles.This method combines visual information with spatial distribution to improve lane line recognition in a complicated environment.First,the grid density of YOLOv3 is enhanced from S×S to S×2S,which targets the points in the bird’s-eye view where the lane lines had different densities in both horizontal and vertical directions.YOLOv3(S×2S)results are better suited for detecting small targets in an image.YOLOv3 algorithm uses the feature pyramid networks(FPN)to make multiple-scale predictions.It uses the residual neural network(Res Net)to identify image features and balance detection speed and accuracy.Secondly,by utilizing the characteristics of lane line serialization and structural correlation,a new lane line detection model based on the distribution of lane lines(Bi GRU-L)is proposed by using a bidirectional gated recurrent unit(Bi GRU).Lastly,a Dempster-Shafer(D-S)algorithm based on confidence was used to integrate the detection results from YOLOv3(S×2S)and Bi GRU-L to improve the ability to detect lane lines in complex environments.A dataset created by the Karlsruhe Institute of Technology and Toyota Technological Institute(KITTI)in Chicago was used for the experiment,while Euro Truck Simulator 2(ETS2)was used as a supplement.The detection results have a high level of accuracy in an environment with complex characteristics based on the fusion of YOLOv3(S×2S)and Bi GRU-L models in the D-S model by 90.28 m AP(mean of average precision).The detection speed is 40.20 frames per second,which enables real-time detection.