Constructing environment maps with an active vision system through information assimilation

This dissertation describes an approach for constructing an environment map by multi-sensory information assimilation. The approach has been implemented using a binocular stereo vision system mounted on a mobile robot. The focus of this work is on the development of a complete system for constructing an environment map, dealing with issues of sensor modeling, data acquisition, data transformation, data registration, data fusion, robustness, and computational tractability. We draw on several fundamental results from estimation theory and robust statistics.; We take the view that the approach used in a given situation should be determined by the application and the nature of the data that can be extracted from the available sensors. We first address the problem of building a coarse map of the environment, in the form of a certainty grid, using dynamic stereo. We present an approach for multi-sensory depth information assimilation based on Dempster-Shafer theory for evidential reasoning. This approach is suited for assimilating depth related information from physically different sensors. This approach provides a mechanism to explicitly model ignorance which is desirable when dealing with an unknown environment and offers some representational advantages over the traditional Bayesian approach. We then deal with the problem of constructing high-resolution environment maps from pixel-level data. The representation sought here is in the form of a dense stereo disparity map which is suitable for highly textured scenes. This representation could be useful for extracting higher level features, in particular surface patches. Finally, we deal with the problem of recovering a wire-frame description of an environment which is suitable for scenes containing mostly polyhedral objects. We present a segment-based stereo matching algorithm designed for dynamic stereo sequences. This algorithm utilizes a belief function based approach to computer a reliable wire-frame description. This representation combined with geometric reasoning and model-based vision techniques could be useful for object recognition tasks.