Efficient Real-time Binocular Camera Tracking And Online 3D Reconstruction

Binocular camera based real-time camera tracking and online 3D reconstruction is a very meaningful research topic in the field of computer vision.At present,more and more mobile terminals are equipped with binocular cameras.Especially in recent years,because of the rise of augmented reality applications,the realization of efficient and stable simultaneous localization and mapping based on binocular cameras has become very practical.For the real-time camera tracking,how to obtain high-quality feature matching and effectively reduce the cumulative error are two very critical issues.We designed an efficient and real-time binocular camera tracking system for these two problems.Based on this,we realized online 3D reconstruction combined with stereo matching and depth fusion technology.The main contributions of this paper are as follows:1)Making full use of the binocular camera can directly restore the depth characteristics through stereo matching and excavate the intrinsic planar structure of the scene.Therefore,we propose a motion prediction and multi-plane fitting to improve the success rate and stability of feature matching.2)We propose to use a thumbnail-based color histogram metric and image alignment method to achieve robust relocation and recover quickly from the state of tracking lost.For loop closure,a method similar to relocation is used to search for historical keyframes with high similarity to the current frame as candidate frames,and keyframe similarity vector matching is used to loop closure,thereby reducing or eliminating error accumulation.3)Based on the real-time camera tracking,it is proposed to further restore the depth map of the key frame by using the stereo matching method,and perform online fusion of the depth map to realize online 3D geometric reconstruction.