3D Object Detect Driven By 2D Image

Object detection has evolved to be very mature today.Relying on powerful deep learning techniques,today's object detection algorithms have moved away from mathematical descriptions and turned into automatic machine learning feature description methods with amazing results.After 2D object detection techniques have been applied to real-world engineering,attention has been focused on 3D object detection.The problem with deep learning,however,is that it relies on a large number of labeled datasets.Data sets for 2D images are easy to obtain,but 3D point cloud data sets are much more expensive to acquire.While 2D object detection can use cameras as image acquisition devices,3D object detection requires depth perception devices to acquire point clouds.Common depth perception equipment such as LIDAR,depth camera,etc..The effective distance of LIDAR is about 150 m,but it is expensive,usually ranging from tens of thousands to hundreds of thousands.The high price of LIDAR limits the practical application of 3D object detection,and the effective distance of inexpensive depth cameras is simply not enough to meet the task of object detection at a slightly longer distance.Therefore,the key problem of three-dimensional object detection: cost and effective distance.For 3D object detection,not only the hardware cost is high,the software cost is also high,and the 3D point cloud data is difficult to obtain and difficult to label,especially the 3D point cloud of the long-distance object is not available at all.However,the cameras we use everyday can acquire rich information and see very long-distance objects at low prices,but due to the camera imaging model itself,the depth is lost during the imaging process and the object depth is not recoverable,and the depth cannot be recovered directly using images.Therefore,in this paper,we take a unique and non-mainstream perspective and investigate how to use low-cost cameras as the main sensing device and depth-aware devices as auxiliary or without any depthaware devices for 3D object detection with low cost and long effective distance,which is of high specific engineering significance.The main innovative research work and findings of this paper are as follows.1)A fast image enhancement method adapted to a variety of scenes is proposed to improve the image quality and enhance the accuracy of detection as a pre-processing work for subsequent object detection.For the problems of high specificity and poor generality of various algorithms for image enhancement,slow calculation speed,and often color deviation after color recovery,this paper proposes a fast adaptive fast image enhancement method,which includes light equalization and histogram correction.An adaptive offset method is proposed for too bright or too dark images.The mean and variance are used to extract the main part of the image for color scale adjustment,and finally the light brightness is extracted using Gaussian filter for light equalization.The whole algorithm has only one convolution operation,which is fast,effective and general.Through experimental verification,the algorithm is effective for underwater,defogging,night vision,overexposure,and excessive color deviation,and it is adaptive and strong without changing parameters.It can be applied among car driving,underwater equipment,and aerospace imaging equipment.2)A method for estimating the 3D position of a object based on instance segmentation is proposed.The method proposed in this work does not require a 3D data training set,allows fast and accurate extraction of 3D object point clouds,and can achieve 3D object detection using only 2D detectors.The method proposed in this paper can be used in a variety of sensors,such as RGBD cameras,camera-radar,binocular cameras,etc.Firstly,the object is segmented by instances under 2D images,the depth image of the object is extracted with RGB images according to the segmentation mask of the object and transformed into a coarse point cloud,and finally anomalous noise point removal is performed to obtain a fine object point cloud.The method is tested on outdoor dataset KITTI and indoor dataset TUM,and the results show that the method can accurately estimate the object location information,is simple and low cost to implement compared with other point cloud segmentation algorithms,and does not require any point cloud data as training samples.3)A datum prior theory is proposed for object depth estimation in monocular cameras.In this work,we propose a simple,fast and effective object depth estimation method called datum prior,which is built into the camera imaging model to provide a constraint for depth recovery in 2D images by finding the intersection of the object of interest with the datum to estimate the position of the object in the camera coordinate system.It is assumed that some common objects(e.g.,cars,people)are standing on the ground and that the normal vector of the ground is constant.The camera height is known,and the position of the object in the camera coordinate system is estimated by finding the contact point between the object and the reference surface.Results: We conducted error experiments on KITTI.there are more than 10 k objects in KITTI,the environment is complex,and most of the objects are between 7m-90 m in depth.through a large number of statistics,the average error of our method on KITTI is between 7%-25%,and the error is positively correlated with distance.Meanwhile,we collected a small number of long-range samples outdoors,with objects between 100m-200 m,and the error of our method is between 10%-30%.The experiments demonstrate that our proposed method can estimate the distance of the object without using any depth-aware device,is fast and low cost,and the effective action distance depends on the camera,using a telephoto camera to reach very long distances.