Interpretation of the three-dimensional visual environment from uncalibrated image sequences

Metric reconstruction of a scene viewed by an uncalibrated camera undergoing an unknown motion, is one of the most fundamental problems in computer vision. Recent years have seen significant progress in reliable analysis of image sequences and the recovered 3D scene information can be used to generate new viewpoints, acquire 3D models, track, insert and delete objects, or determine the ego-motion for visual navigation.; Inferring information about a scene starting from an image sequence is a difficult task and the usual approach is to divide the problem into several manageable subproblems. The processing stages are composed from lower-level tasks such as extracting salient image features to higher-level tasks such as determining camera positions relative to the viewed scene. The reliability of the processing chain depends on the robustness of each module and the ability to cope with incorrect or noisy measurements. In this thesis we have redefined several of the processing modules and developed a highly accurate and robust system for the recovery of the 3D visual environment.; The process of image filtering is reformulated in a linear vector space and the role of different subspaces is analyzed in the context of edge detection. An edge confidence measure is introduced which allows higher sensitivity to sharp but weak edges. Based on the distribution of image edge points in the line parametric space, a method for lens distortion correction is presented.; For the detection of interest point correspondences we have combined the traditional optical flow with matching color distributions. Oriented kernels are introduced in the spatial domain to compute the color distributions, thus obtaining rotation sensitivity. A joint robust minimization procedure is employed and subpixel accuracy is achieved under large image transformations.; Estimation of the structural and camera parameters relies on bundle adjustment, a nonlinear optimization technique which minimizes the reprojection error. The initial solution is usually obtained by solving a linearized constraint at each stage of the reconstruction process. The traditional way to obtain the initial solution is to apply a total least squares (TLS) procedure which yields a biased estimate because it fails to correctly account for the noise process that affects the linearized measurements. We present a more balanced approach where the initial solution is obtained from a statistically justified estimator which assures its unbiasedness. The quality of this initial solution, obtained using the heteroscedastic errors-in-variables (HEIV) estimator, is already comparable with that of the bundle adjustment output, and thus the burden on the latter is drastically reduced while its reliability is significantly increased.; Each module is tested on synthetic data and standard images and the performance of the 3D reconstruction system is illustrated on several uncalibrated image sequences.