Research On 3D Reconstruction From Image Sequences

For a given set of images, the goals of 3D reconstruction are to recovery both the poses and positions of all the cameras, and surface of the spatial scene. Since last 30 years, broadly discussions and deeply research have been conducted, and the achievements have been applied to resolve many practical problems mostly from medicine system and digital entertainment system. The following creative studies of this paper are around the task of "3D Reconstruction from Image Sequences", and focus on theories, implementations and applications related to camera self-calibration, stereo vision and multi-view surface reconstruction:(1) In order to dispose of camera self-calibration in spite of freely varying and unknown internal and external camera parameters, an efficient new approach without anything known about the scene and the cameras is proposed, and the reconstructed results for the uncalibrated image sequence are given. It can handle the problem that present algorithms cannot deal with cameras with aspect ratio deviated from 1.0 well. Compared with existing algorithms, both intrinsic and external parameters of cameras and feature points of the scene can be recovered accurately and efficiently, even for the cameras with aspect ratio greatly deviated from 1.0, like 0.6 and 2.3 in practice, through skillfully handling intrinsic parameters in the cost function used to handle the variables.(2) Image matching in stereo vision, namely stereo matching or stereo correspondence, requires to compute each pixel's disparity accurately especially in the areas with sparse textures, patterns, discontinuities or occlusions. For the images pair with not large resolutions, in order to estimate disparities of all the pixels in those areas at the same time, an efficient algorithm based on graph cut is proposed. It mainly consists of 2 steps:pixel matching cost computation and global optimization of the disparity map sequentially. The first step adopts a special pixel matching algorithm with adaptive weights, which is insensitive to image sampling, so that both over-smoothing problems in discontinuities and disparity errors in sparse textural areas caused by current methods can be sharply reduced. The second step can explicitly integrate both occlusion and discontinuity costs into the energy functions to regularize the disparity map, and the optimum can be solved rapidly by graph-cut based energy minimization.(3) For a pair of images with high resolution, the general global algorithms are difficult to converge to the global optimal value and it is time cost, and then a part of pixels' disparities can be calculated fatly. To improve this time efficiency, a new stereo matching algorithm with an adaptive window and weights based on confidence is proposed, which has synthesized both advantages of adaptive window algorithm in discontinuities and adaptive weights algorithm in sparse textural areas. This algorithm firstly shapes the matching costs of pixel pairs, and then aggregates the costs with the adaptive weights supported and fixed size window. Secondly, it finds out the matches with the minimum cost (i.e. WTA optimization), and calculates the confident values of which according to the confidence formula. If the value is larger than the predefined threshold, then the match is reliable and output the resulting disparity map.(4) Compared with sparse reconstruction, multi-view surface reconstruction is to recover the close surface of the scene. Due to the limitations of the number of the images and viewpoints of the cameras, more than one scene is possibly compatible with a set of images. In order to reduce ambiguity as much as possible, a local photo-consistency constraint is defined on one hand to constrain the local color distributions of the projected pixels from the same 3D point, and a local smoothness constraint on the other hand to produce a more smooth surface, then construct a global energy function, and resolve the optimization fast through graph cut based method. A survey has been done, and an implementing algorithmic model has been provided, where several representative algorithms have been implemented and compared, while part experimental results have also been given.