Research And Application Of 3D Object Detection Based On Pseudo-lidar

In recent years,with the rapid development of autonomous driving and robotics,the need for realizing a robust and high-performance 3D object detection system has become more and more urgent.In detail,3D object detection system takes 3D environmental information as input and outputs the category of objects as well as their length,width,height,rotation angle and other information in 3D space.In practice,the mainstreams methods are lidar-based and visionbased ones.Though lidar-based methods can provide high-precision 3D environmental information,but they are expensive,making the cost of landing autonomous driving technology increase;on the other hand,the vision-based ones are easy to install and low-cost,but they cannot provide accurate depth information and are susceptible to environmental impacts,which leads to a much lower accuracy rate of vision-based methods than lidar-based methods.To solve the above problems,Pseudo-Lidar provides a low-cost alternative that can effectively bridge the gap between vision-based and lidar-based methods.In general,PseudoLidar is a deep learning technique that predicts the depth value of each pixel in an image by a monocular or binocular depth estimation algorithm and converts the depth maps to PseudoLidar point cloud data using the coordinate relationship between the camera and the lidar in space,where the Pseudo-Lidar point cloud can be applied to any lidar-based 3D object detection method to enhance the detection accuracy.Motivated by this end,we in this thesis focus on the study and application on Pseudo-Lidar-based 3D object detection framework.The main work is as follows:1.Investigate the architectural composition of the Pseudo-Lidar-based 3D object detection method,which consists of three modules: depth estimation to generate disparity maps,transformation of depth maps to Pseudo-Lidar point cloud data,and 3D object detection using pseudo point cloud data.For depth estimation to generate disparity maps,the process of feature extraction,feature matching and its regularization and disparity calculation are studied;for transformation of depth maps to Pseudo-Lidar point cloud data,the coordinate transformation relationship is obtained based on the relative positions of camera and lidar in 3D space to generate pseudo point cloud data;for 3D object detection,the generation of 3D region proposals in the first stage and the correction process of region proposals in the second stage are studied.2.To address the problems of large noise,excessive redundant points and inaccurate localization in the original Pseudo-Lidar framework,a pseudo point cloud generation method based on optical flow stereo matching is proposed.Firstly,the disparity maps are generated by an optical flow-based stereo matching network,which combines optical flow matching with stereo matching to reduce the computational cost as well as to improve the accuracy of disparity prediction.Then,in the training phase,a multimodal synthetic dataset is used to enhance the network stability.Finally,a depth correction method based on KNN graph is used to correct the predicted depth maps using real sparse point cloud data.3.An efficient Pseudo-Lidar-based binocular 3D object detection method,namely,PLRSPV,is proposed.In detail,the input of the method is binocular images and sparse point cloud data,and the pseudo point cloud data is first generated by the optical flow stereo matching module.The pseudo point cloud data is then used as the input to be fed into a joint voxel and point based 3D object detection method.As a result,this method has combined the efficient computation of voxel-based methods and the rich information of point-based methods,which can greatly enhance the detection accuracy.In summary,based on the research of Pseudo-Lidar-based 3D object detection framework,this thesis proposes a pseudo point cloud generation method based on optical flow stereo matching,and an efficient Pseudo-Lidar-based binocular 3D object detection method,which is a greatly improved one in terms of accuracy and real-time performance and effectively narrow the gap between vision-based and lidar-based 3D object detection methods.