3D Object Detection And Optimization Based On Depth Sensor And Color Camera

The main purpose of the object detection task is to determine the category of the objects in the scene,and use 3D boxes to mark its three-dimensional position and size.At present,deep learning can obtain models by learning labeled data so that can extract more robust features.Although it can achieve good results,the challenges are also very large.The reasons include errors in the input information,errors in extracting features,and so on.Therefore,solving the above problems is also an important guide to promote the detection accuracy of 3D object detection technology.This paper aims at some problems exiting in 3D object detection tasks such as: lack of dependent relation in region and channel features extracted by convolutional neural networks,difficulty in expanding the receptive field without losing information,low detection accuracy and high miss detection with one stage method,sparse and incomplete point cloud,and so on.A series of solutions are proposed at enhancing the robustness of features extracted by the convolutional neural network and optimizing the 3D proposal boxes,the research content mainly includes the following parts:1.Aiming at the features extracted by the convolutional neural network lacking the dependencies between different regions and different channels,this paper proposes a convolutional neural network that combines mixed domain attention mechanism to extract features.Firstly,a self-supervised spatial domain attention mechanism is incorporated into the network to transform the spatial position of the objects in the input information.Then channel domain attention mechanisms are incorporated into each convolutional layer of the network to enhance key channel characteristics and suppress invalid channel characteristics.Finally,combine different dilated convolutional layer to extract the features of different receptive fields so that the features of the global and local vision of the image are obtained.2.Aiming at the problem that the features extracted by the convolutional neural network are difficult to ensure the expansion of the receptive field without loss of spatial resolution,this paper proposes a convolutional neural network that combines multiple layers of dilated convolutional layers to extract features.Firstly,the features extracted from the network are input into the combined dilated convolutional layer to extract the feature maps of different receptive fields.Then the channel domain attention mechanism is incorporated to obtain the channel weights of the feature maps and fuse them,therefore,while obtaining the key channel features of the feature local receptive fields,it also gets the key information of its global receptive field.3.Aiming at the problems of low accuracy and high miss rate detected via one stage 3D object detection method,this paper proposes a two-stage 3D object detection method,which based on feature pyramid structure to improve the accuracy and miss rate of 3D object detection.The experimental results show that our method has higher detection performance than some out-of-state 3D object detection methods.4.Due to the incomplete information of LIDAR point cloud,the regression boxes of 3D object detection have some errors.To alleviate the problem of inaccurate detection results due to input information,this paper proposes a 3D object detection method that introduces point cloud shape completion optimization.Firstly,the point cloud shape completion algorithm is used to complete the shape of LIDAR point cloud of the object extracted from the 3D bounding box,which was obtained by 3D object detection network,so that the complete shape LIDAR point cloud of object are obtained.Then use principal component analysis algorithm to extract the feature of the complete shape LIDAR point cloud,so that a more accurate 3D box is obtained.