Deep Learning 3D Object Detection

3D object detection is one of the more important tasks in computer vision,and it has important applications in automatic driving,robotic arm grasping,augmented reality and so-on.3D object detection on point clouds is more challenging than the 2D object detection on images.The sparse and irregular attributes of the point clouds ask for a higher requirement for the algorithm design.In this thesis,we focus on the algorithm of deep learning 3D object detection.The main contents are as follows:(1)We propose a novel method termed Frustum Conv Net(F-Conv Net)for amodal 3D object detection from point clouds.Leveraging current mature 2D object detectors to provide frustum proposals,we use a novel grouping mechanism – sliding frustums to aggregate local point-wise features.We transform unordered and irregular point clouds into regular feature maps for use of a subsequent fully convolutional neural network(FCN).Our proposed FConv Net achieves competitive results on both the indoor SUNRGBD dataset and the outdoor KITTI dataset,surpassing all methods in the same period.(2)We extend F-Conv Net to 3D object detection under binocular(stereo)vision setting.In this setting,we only use two color camera images as input and don't need any depth data such as Li DAR point clouds as input.Compared with current existing methods,our proposed F-Conv Net shows obvious advantages.(3)In view of the fact that current 3D object detection relies too much on 2D object detection,and 2D detection is difficult to detect occluded objects,we make an improvement on the 3D object detection algorithm Vote Net which does not depend on images.We propose the second extraction and aggregation of point clouds and features to achieve better boundary prediction.Compared with the baseline,we effectively improve the performance of 3D object detection on both SUNRGBD and Scan Net dataset.(4)Due to the weak feature extraction ability of Point Net++ network,we design a semantic-aware 3D object detection by taking the semantic category predicted by submanifold sparse convolution neural network UNet as extra input of each point.Compared with the benchmark method,the performance of 3D object detection on the Scan Net dataset has been greatly improved.(5)We extend object detection to more refined tasks – instance segmentation and panoptic segmentation.We design heuristic fusion algorithms to integrate the results of 3D object detection and semantic segmentation and achieve great performance on both point cloud instance segmentation and panoramic segmentation tasks of Scan Net dataset,outperforming all the published papers in the same period.