3d Reconstruction From Multiple Cameras And Its Applications

3D reconstruction from multiple cameras is a hot problem in computer vision, and has received a significant attention in the past two decades. Although fruitful achievements have been obtained for this problem, there still remain many theoretic and technical problems to be solved. This thesis focuses on some key aspects of 3D reconstruction from multiple cameras, such as multi-camera calibration, visibility estimation, multi-view reconstruction, etc.This thesis proposes a grouping-based method for multi-camera calibration. The method can determine both the intrinsic and extrinsic parameters of cameras in a uniform world coordinate system. We first divide the cameras into groups according to their positions and orientations, to ensure that all the cameras in each group are convergently placed, and then estimate the intrinsic and extrinsic parameters of each camera in the coordinate system of its own group. Common views of planes between groups are used to position each group to a uniform world coordinate system. By dividing the cameras into groups, the method can be used to calibrate multi-camera system with non-convergent configuration. When recovering the rigid displacements between cameras, all the common views of planes are taken into account instead of selecting only one of them randomly. This makes the method possible to handle the inconsistency problem when estimating the extrinsic parameters of cameras. Experimental results show the validity of the method.Image-to-image matching is the central aspect of the techniques for 3D reconstruction from multiple cameras. It is necessary to determine the visible regions of the given cameras in a 3D scene for solving the occlusion problem in the matching process. This thesis presents a visibility algorithm which is formulated in an implicit framework to estimate the visibility of multiple cameras. In this algorithm, the scene geometry is implicitly represented by a level set function, and the visibility of each point in the scene with respect to the given cameras is determined using an implicit ray tracing technique according to the global scene geometry. In contrast to some other visibility algorithms based on the explicit representation of the scene geometry, our algorithm is more efficient because the time-consuming calculation in the ray tracing process is eliminated. Experimental results in the synthetic scene support the algorithm.This thesis also describes an evolution-based approach for multi-view reconstruction. The approach relies on a variational level set framework, where a variational energy functional that includes both the photo-consistency and the region-consistency constraints is defined. We cast the multi-view reconstruction problem as an optimization of the energy functional amenable for minimization with the variational principle, and the final reconstruction result is a minimizer of the energy functional. Compared with the multi-view reconstruction methods based on traditional level set framework, our approach has the following advantages: (1) The minimization of the energy functional can lead naturally to the evolution equation of the level set function; (2) The final result is not sensitive to the initial value of the level set function; (3) The costly re-initialization procedure of the level set function can be completely eliminated, and the numerical implementation is thus simple and efficient; (4) It is convenient and natural for incorporating additional information into energy functional, and therefore produces more robust results. Experimental results on the middlebury dataset and real images demonstrate the effectiveness of the approach.The proposed methods have been applied to human face reconstruction, and a multi-camera human face reconstruction system consisting of multiple stereo heads has been developed for modeling realistic looking human faces.