Real-time Monocular Depth Estimation And 3D Reconstruction

3D reconstruction technology is an important research topic in computer vision and graphics.It has been widely used in many fields such as virtual reality,augmented reality,intangible cultural heritage protection,film and television games.The process of 3D reconstruction based on a monocular camera can be divided into two modules:one is calculating the globally stable and consistent camera pose in real time.The other is recovering dense point cloud and fusing global model.During camera pose estimation which is based on image keypoints,a large number of calculations will be introduced in feature detection.And Estimation of dense depth map often requires high computational cost.The camera pose estimation module of our 3D reconstruction in this paper adopts a stable and robust simultaneous locatoin and mapping system:ORB-SLAM.In order to speed up keypoints detection,we optimize the keypoints detection of ORB-SLAM.We extract ORB keypoints evenly and filter keypoints by quadtree.Such keypoints detection enhances the robustness of tracking and reduces the absolute error of trajectory.The major shortcoming of traditional contructing tree methods is that they construct levels of the tree sequentially,which limits the parallelismability that they can achieve.We implement maximizing the parallelism of constructing each node of binary radix tree,and it can be transformed into quadtree,octree,and KD tree.In the depth estimation module,we select an appropriate frame in the ORB-SLAM keyframe buffer pool as reference of current keyframe.Then we use this pair of keyframes to recover dense depth map by a local-based dual-view depth estimation method which recovers depth maps fastly by GPU.Furthermore,we produce point clouds from those depth maps.Finally,scene reconstruction and model fusion are performed based on keyframe pose and point clouds.The biggest challenge of this module is to estimate dense depth map in real time.Compared with the original ORB keypoints detection in OPENCV,our keypoints detection reduces the pose estimation error and improves the robustness of SLAM system.Compared with the original keypoints detection in the ORB-SLAM,our method obtains different degrees of speed increase under different parameters.Our death estimation algorithm is able to recover depth map of monocular image in real time.Combined with camera pose estimation and keyframes of ORB-SLAM we can obtain a pretty nice 3D model.