Real Time Object Detection And Localization Based On Deep Learning And Multi Camera Fusion

While 2D object detection from monocular images has made significant progress in the past few years,the output is more on a semantic understanding of the real world.However,such 2D understanding is less ideal for interactions with the real world,which is inherently 3D.The accurate perception of our surrounding 3D space helps us navigate through 3D space while avoiding obstacles.Such 3D perception capabilities,especially 3D detection and localization,are vital for autonomous driving as well — the application we are particularly interested in in this paper.Based on SSD[6],we propose SS3D,a single shot 3D detector that takes an monocular image as input,and outputs oriented 3D bounding boxes for detected objects.The 3D detector is realized by lifting estimations of 2D bounding box,pose and dimension into 3D,via specific geometric constraints.Beyond the 2D-3D lifting,we extensively studied SSD proposal space,and introduce a method to match it with the data distribution for effectively training.To enhance SS3D's performance on hard cases,we propose to synthesis hard positives by adding more occlusions to training samples.Both proposal distribution matching and hard positive synthesis methods are simple,yet shown to be quite effective.Extensive experiments and comparisons on KITTI dataset are presented to demonstrate the performance of our SS3D.