Real-time 3D Object Detection And Tracking System For Augmented Reality

3D object detection and tracking refer to detect and then track the pose of one or more designated 3D objects of a continuous sequence of images.We can provide information for the interaction of virtual and real objects in augmented reality,provide the basis for the task planning of the robot's capture,and provide a reference to positioning between vehicles based on the detection and tracking results of 3D objects.Therefore,3D object detection and tracking are very important in the field of augmented reality,robot and automatic driving.Essentially,3D object detection and tracking is the perception of objects exist in the real world.The perception can be divided into two levels.The first level is to check whether the designated objects exist on the environment and find the position of the object,which is achieved by object detection.The second level is to compute objects' pose of successive time domains,which is achieved by trackingAt present,3D object detection and tracking are still in the development stage in the industry.Compared with mature 2D object detection and tracking,there are many key problems to be solved in the field of 3D object detection and tracking,for example,the diversity and complexity of 3D object makes detection and tracking prone to fail,and lots of parameters involved by algorithms causes low efficiency.By deeply studying the current attempts made by the academic about detection and tracking of 3D object,we propose a feasible and efficient 3D object detection and tracking system based on the knowledge of neural networks and the SLAM system.The main contributions of this paper are as follows:1)Some current mainstream methods are always time-consuming in the detection stage because of the complexity of 3D object and lots of parameters involved by algorithms.We proposes a new neural network model,which is much more efficient than most mainstream methods,and completes real-time 3D object detection on the PC side.What is more,we uses the realistic rendered data as training data onto greater generalization ability.2)Compared to 2D object,3D object,which always is textureless or in a complex environment,have lower accuracy of detection.The 3D object detection method implemented by us redesigned a set of detection logic,using the 3D bounding boxes of the objects and the 3D pose information as the training data for detection,and then using the edge information of the objects to optimize the detection result,finally,achieved a comparable mAP(mean Average Precision)to the state of the art methods.3)Mainstream 3D object tracking methods is difficult to keep stable tracking in complex environments.In order to solve this problem,we combine 3D object detection and SLAM system.The tracking of the objects is completed by SLAM,which ensures the continuity and robustness of the tracking.At last,we implement augmented reality application in iOS system.The experimental results show that the 3D object detection and detection system of this paper has high efficiency,accuracy and practicability.