Image-based 3D Scene Reconstruction

How to use the information of images to reconstruct the scene in real world is a hot spot in computer vision, the theoretical results are now widely used in measurement, virtual reality, movies, entertainment, and cultural heritage protection, etc.3D reconstruction technique involves theoretical methods of many fields including feature extracting, image matching, multi-view geometry theory, numerical analysis, computer graphics and so on, what's more, any part is crucial to the final result.We have developed an image-based 3D scene reconstruction system that can automatically reconstruct the 3D scene with unordered images from common digital cameras. In order to meet the requirements of different applications, the system is able to output sparse point clouds, dense point cloud and 3D model. In the paper, we do some research on the main four modules of the system, that is feature detecting and matching module,3D structure recovery module, dense point cloud reconstruction module and surface reconstruction module.Firstly, we study the features detecting, matching, and tracking. Then we adopt the SIFT algorithm to improve the image data's invariance to rotation, scale and illumination change. Meanwhile, we track the feature points quickly through image connection diagram.Secondly, we study the 3D structure recovery method. Epipolar geometry and global optimization algorithm are used to recover cameras'parameters and get the sparse points cloud of 3D scene. To simplify the process of reconstruction, we achieve for the self-calibration of the cameras with the EXIF tag information of images as the initial value.Thirdly, we focus on the dense points cloud method. We mainly explain the PMVS (Patch based Multi-View Stereopesis) method including concepts of patch model, photometric discrepancy function, patch optimization and image model, and give the implementation of PMVS. PMVS algorithm can be used to obtain the dense points cloud of the surface of scenes, which satisfys the requirements of the project.Finally, we study the surface reconstruction for the 3D scene. Poisson surface reconstruction algorithm can infer the topology of surface, filter noise data, fill holes legitimately, adjust the sample data, and divide model network, so it can better cope with the problem that the dense point cloud generated by images is scattered and uneven distribution. We implement the Poisson surface reconstruction algorithm, and verify the feasibility by experiments.