A 3D trinocular active vision system for surface reconstruction

Representing the world as 3D objects improves the performance of many vision research areas such as object segmentation, object recognition, and robot navigation. 3D laser scanners and stereo imagery have been used to build a 3D representation of the environment. Use of the laser scanner is restricted to a small range. It is generally agreed that individual stereo modules are prone to errors and often ambiguous when used in real-world applications. Thus, there exists much interest in the design of integrated vision systems that perform reliably in practical situations. The design of such systems is challenging, because each module works under a different and possibly conflicting set of assumptions.; In this thesis, we present a prototype for a 3D trinocular active vision system that can be used reliably in real-world applications. The system uses an agile trinocular camera head that can be mounted on a mobile robot or on a robotic arm. A major function of the system, henceforth called "CardEye", is surface reconstruction by recovery of depth information. An active lighting device working in two operating modes---range finder and pattern generator is used to assist the recovery process.; Furthermore, we describe a new technique for solving the sensor planning problem of Multi-camera vision systems. This allows the automatic selection of camera, parameters. The technique employs the fixation point of the system to generate the sensor parameters (vergence, baseline, zoom and focus) that satisfy different vision constraints (field of view, focus, disparity and overlap). We derive a set of equations for these constraints and use an optimization method to provide a closed-form solution for the translation between the three cameras. In this way, the vergence angle of the cameras as well as zoom and focus setting are determined.; We propose a new multi-stage reconstruction algorithm. This technique integrates edge-based stereo and area-based stereo to combine the accuracy of the former and the richness of the latter and employs structured light to reconstruct featureless and smooth objects that cannot be properly handled with edge- or area-based stereo. The integration is performed by reconstructing the actual and induced edges in the scene using the geometrical constraints of trinocular vision, followed by applying the continuity and the epipolar constraints to grow the surface in the vicinity of the reconstructed edges. Our approach demonstrates that the integration of the three techniques: structured light, edge- and area-based stereo enables the system to handle a variety of surface characteristics.; Potential applications of the CardEye include, but are not limited to, 3D model building, object recognition, object tracking, and robot navigation.