3D Reconstruction Based On Video

3D reconstruction based on video has play an important role in computer vision, and how to recover 3D scene model has been paid much attention and is a difficult problem. Based on the importance of 3D reconstruction, in this paper, the 3D reconstruction based on video has been studied, including 3D scene reconstruction based on monocular video and depth map and scene flow estimating based on binocular video. Since the monocular contains much less depth information, how to recover the camera motion and depth map has been a difficult problem. Besides, although binocular view contains significant depth information, it is difficult to keep the consistency of depth map and moving objects. Therefore, in view of the problems mentioned above, the specific research works are as follows:First, we introduce two directions of 3D reconstruction in computer vision: based on stereo vision method and based on structure from motion. The stereo matching method has been introduced in detail, including algorithm principle, classification, and evaluation method. And, we compare the global, local and semi-global algorithm on four typical dataset. In addition, we have made a detail introduction of structure from motion(SFM), and the experiment has been carried out to get 3D point cloud.Second, a method for depth map and scene flow estimation is proposed. First, input binocular video, initial disparity map is got by SGM, 2D point trajectories are got by optical flow. Then the 3D tracks are got by disparity map and 2D point trajectories, get the object motion hypothesis. Considering constraint between the reference image and the forward-backward images, the energy model based on super-pixel and object is constructed using slanted plane model. Finally, the depth map and scene flow will be got.Third, a method for reconstructing monocular dynamic scene with multiple moving rigid objects captured by a single moving camera is proposed. First of all, feature points are matched through the video sequence via the optical flow method and the “tracks” are got based on these matches. Then the “tracks” are divided into several groups according to their motion differences. An improved graph cuts based multi-label auto image segmentation method is used to acquire the accurate boundary of each moving object and the static background. Then we assume a virtual camera for each moving object and the static background. The pose of these virtual cameras are estimated via the standard Structure from Motion(SFM) pipeline. Finally a dense point set and textured model is returned for each virtual camera. We evaluate our approach on real-world video sequence and demonstrate its robustness and effectiveness.