3D Object Detection And Optimization Based On RGB And Depth Information

In recent years,emerging technologies such as autonomous driving,intelligent robots,and virtual reality have attracted much attention.As one of the key technologies in these emerging fields,3D object detection has received widespread attention.The core task of3 D object detection is to determine the category of the object and the position,size,and orientation angle of the 3D box of the object in the 3D stereo world by computer vision technology.At present,3D object detection based on RGB images only,because of lack of depth information,convolutional neural networks cannot extract high-dimensional spatial features of the scene.3D object detection based on depth information only,because of lack of color,texture information,convolutional neural networks is hard to extract color and texture features.It can be seen that the object contains color,texture information,and depth information.Missing one will make it difficult to describe the category of the object and the position,size,and orientation angle of the 3D frame of the object.In addition,there are often occlusions in the real world,depth cameras are susceptible to instability due to changes in light,and radar point clouds usually have sparseness and noise.Therefore,3D object detection tasks face huge challenges.Therefore,it is particularly important to detect3 D objects based on RGB and depth information and adapt them to realistic application scenarios,which will greatly advance the pace of artificial intelligence technology.Based on the analysis of various 3D object detection methods in related fields,this paper aims at the problem of 3D object detection and summarizes its advantages and disadvantages.Then,starting from the improvement the depth information utilization and the reduction of the calculation complexity of the network model and restoration of the lost depth information of the occluded area of the object,the main research contents include the following parts:(1)Completing the sparse radar point cloud into a dense depth map and eliminatingnoise.For outdoor applications such as autonomous driving and ground-based intelligent robots,the depth camera is subject to instability due to the effects of changes in light,so radar cameras are more suitable for outdoor applications.However,radar point clouds often have sparseness and noise,which are prone to interfere with 3D object detection tasks.Therefore,this paper introduces depth completion algorithm,which complements sparse radar point clouds to dense depth maps and eliminates noise for 3D object detection,so as to improve the depth information utilization and avoid noise interference.(2)Combine the instance segmentation algorithm to narrow the 3D space search range of the 3D object detection network.This paper studies 3D object detection based on RGB maps and radar point clouds.For large-scale scenarios such as autonomous driving,the number of point clouds is huge.3D spatial search of point clouds will lead to excessive computation and complexity of convolutional neural networks.Therefore,in this paper,an instance segmentation algorithm is used to narrow the 3D space search range of the 3D object detection network.(3)Combine the image repair algorithm to repair the missing depth information of the object occluded area.The existence of occlusion leads to the loss of color and depth information of the occluded part of the object,which results in incomplete 3D information of the object and will interfere with the 3D object detection task.Therefore,in this paper,traditional image restoration algorithms are used to restore the lost depth information where the object is occluded,so as to provide more complete 3D information for the 3D object detection network.(4)3D object detection based on 3D information.The 3D object detection method based on 2.5D information lacks the use of 3D information,this paper uses 3D information for 3D object detection,that is,using color cameras and radar cameras to obtain RGB images and corresponding radar point clouds,and then determine target area of the object on RGB images,after extracting the radar point cloud of the corresponding area,the target point cloud is input to the convolutional neural network to extract features,and finally the3 D frame of the object is predicted through classification and regression networks.