Positioning Algorithm Research Based On 2D-3D Fusion Object Detection

3D Object detection and positioning is basis to human perception.It has been used in many fields such as robotics,autonomous driving,security monitoring,industrial production,intelligent agriculture and so on.3D object detection and positioning needs precise range information,and it has been heavily dependent on expensive equipment such as LiDAR for a long time.According to the theory of 2D object detection and stereo vision ranging,this paper deeply studies the problem of 2D/3D object detection and positioning by using image processing,stereo matching,deep learning and other methods.A fusion object detection and positioning algorithm has been proposed.Experiments and tests of this fusion algorithm are carried out respectively on the open source dataset and the self collected dataset.In the end,the main factors and improvement schemes that affect 3D object detection and positioning are discussed.The main innovations and results are as follows:(1)The theory and method of 2D object detection were combed through.The framework of several representative algorithms were elaborated.The pros and cons of different algorithms were qualitatively analyzed and compared.Through the design of data processing scheme,the algorithm and pre training model of YOLO and Mask R-CNN were applied to the same open source dataset,and were trained and tested.The results showed that the inference speed of YOLO was very fast,which was suitable for real-time detection;the average precision of Mask R-CNN was higher,which also had good segmentation results.Finally,it was selected as the basic algorithm of the 2D detection part.(2)The principle and method of 3D object detection were elaborated.Detailed analysis of various excellent algorithms' processes and network structures.The Frustum-PointNet and PointRCNN were re-implemented.Both performance were tested.Compared with PoinRCNN,the accuracy of proposal bounding boxes extraction in Frustum-PointNet was limited.In this paper,the proposal bounding boxes extraction algorithm of Frustum-PointNet had been improved,and a proposal region extraction algorithm based on 2D instance segmentation was proposed.(3)An object detection and positioning algorithm based on 2D/3D fusion was proposed.The stereo matching model of stereo vision was trained by neural network.The accurate disparity estimation was obtained by inferencing on the KITTI dataset.The pseudo-LiDAR point cloud was generated from the disparity map.After 2D to 3D coordinate transformation and outlier removal,foreground and background were successfully segmented.Two proposal box extraction algorithms were designed to refine the network input.Finally,the 3D object detection network was adjusted.The whole algorithm was tested on KITTI dataset,and the result was equivalent to PointRCNN.The feasibility and validity of the algorithm were verified.(4)The data collection scheme of autonomous driving scene was designed.Fusion object detection and positioning algorithm was applied through the self collection data set.The results showed in terms of vehicle scale prediction,the error was at decimeter level;in terms of coordinate prediction,the error was at meter level.The accuracy of depth estimation played a decisive role in the final results of the 3D detection and positioning task under stereo vision.Therefore,camera calibration,stereo matching and other factors that directly or indirectly affect the effect of depth estimation should be focused.