Research On Self-supervised 3D Object Detection Based On Stereo Images

The 3D object detection methods based on deep learning need large-scale labeled data to achieve start-of-the-art performance.However,the labeling of 3D bounding boxes requires the annotator to adjust the boundary repeatedly based on experience under incomplete 2.5D sparse point cloud,which is very difficult and easy to introduce annotation error.To solve this challenging problem,this paper does research on self-supervised 3D object detection methods.The main content is divided into the following three parts:(1)To address the dependence of 3D object detection on large-scale data and the labeling problem,we propose a novel self-supervised stereo 3D object detection network based on "predict-render-compare" structure.Specifically,the prediction network outputs the 3D parameters of the object.The rendering module reconstructs the model based on the predicted parameters,and renders the point cloud,segmentation mask and depth map of the model.The comparison module computes the comparing loss between the rendered outputs and the 3D point cloud,segmentation mask and depth map of the real scene.The losses back propagate to the prediction network through the differentiable rendering,guiding the learning of the prediction network.Durance inference,only the prediction network is needed.In addition,the improvement strategies for those modules are proposed,including the output improvement strategy,gradient disentangling,loss function and so on,which improve the performance of our network.(2)In order to improve the practicability and competitiveness of our self-supervised 3D detection network,the practical application of self-supervised 3D object detection is further studied.In view of the lack of labeled segmentation and 2D bounding box in the actual automatic driving system,we replace them by the predicted segmentation and 2D bounding box,making the network completely free from annotation.Only need to drive the car,the collected data is enough to train.Aiming at the interference of background,ground and occlusion in actual scene,we design the background filtering algorithm,foreground ground filtering algorithm and foreground occlusion filtering algorithm to deal with point cloud clutter caused by inaccurate predicted mask.In addition,because of the sparsity of 2.5D point cloud and the ambiguity of the angle in the rendered results,the angle learning is easy to fall into a local minimum in application.To deal with this problem,we propose a global angle search algorithm.Based on the feature that the chamfer distance can measure the match level between the rendered model and the real scene,multiple discrete angles are searched globally.We choose the angle that best matches the model and scene as the real angle to guide the prediction network.(3)In order to verify the competitiveness of our self-supervised 3D object detection algorithm,experiments are carried out on the KITTI public dataset,and compared with other fully supervised and self-supervised 3D object detection methods.The performance of our stereo self-supervised detection method is close to that of fully supervised stereo 3D detection methods,which is equivalent to that of self-supervised 3D detection methods based on optimization.And the proposed method can achieve real-time speed,which is ahead of optimized methods.In addition,self-comparison experiments are carried out to verify the effectiveness of the proposed scene filtering algorithm and global angle search algorithm.Through the scene filtering algorithm and global angle search algorithm,the performance of the self-supervised 3D detection method using prediction segmentation is close to that of using manual annotated segmentation.