3D Object Detection And Recognition Based On Deep Neural Networks

Computer vision is an scientific discipline on making machines to undertand the world with sensors and computers,and it has become an important research field in the era of artificial intelligence.Compared with 2D images,3D data exhibit several advantages,it can provide more geometric information and is not affected by variations in illumination and texture commonly.With the development of 3D data acquisition technology,computing power,deep learning and incresing application demand,research and application on 3D computer vision have attracted more and more attention.3D object detection and recognition are key technologies of 3D scene understanding,and the foundational technologies to let machines understand and interact with the world,demonstrating promising application prospects in intelligent unmanned system,AR&VR,remote sensing mapping,biomedical,entertainment multimedia,battlefield perception and so on.In recent years,3D object detection and recognition has become a hot topic in the 3D computer vision.This thesis conducts deep theoretical and technical researches on 3D object detection and recognition,and gains several achievements.For 3D object detection,the research are conducted in two aspects based on singlesource data and multi-source data.For 3D object detection based on only point cloud,a column-based algorithm to learn bird eye view feature maps from point cloud is first proposed,which learn BEV feature maps with PointNet++ based networks in an end-to-end manner.Then a multi-scale feature extraction network is proposed to detect 3D objects with BEV feature maps.The proposed algorithm is highly computational efficiency and performs well for small 3D objects detection.For 3D object detection based on both point cloud and image,a projection and neighborhood interpolation based method,which utilizes the geometric correspondence between 3D point cloud and 2D image,is first proposed to fusion features of point cloud and the corresponding local features of image,and then a 3D object detection network called F-FusionNet is constructed further.Besides,a loss function to exploit the correspondence between 2D and 3D bounding box is proposed to optimize the learning of the detection network.For 3D object recognition,the research are conducted in two aspects based on multiview representation and volumetric representation.For multi-view based 3D object recognition,an algorithm using joint convolution neural network and bidirectional LSTM to learn the feature representation of 3D models is proposed to exploit the relationship between multi-view images,which can improve the discrimination of feature representation of 3D models.Besides,a training method for joint network is designed.For volumetric based 3D object recognition,a methd to train binary volumetric convolutional neural networks for 3D object recognition is proposed to address the high computational and memory cost in 3D volumetric CNNs.The proposed method transforms the inputs and weights in convolutional and fully-connected layers to binary values,so that the volumetric CNNs can be potentially accelerated by efficient bitwise operations.