A differential geometric approach to computer vision and its applications in control

As an important feature of any autonomous mobile agent, such as the human or unmanned (ground and aerial) vehicles, there is usually a vision system embedded within the decision making loop. The role of the vision system, whether biological or artificial, is responsible for retrieving 3D information of the environment from 2D images. Such 3D information contributes to either low-level feedback control so as to safely navigate within and interact with the surroundings, or high-level decision making so as to reliably recognize, evade, pursue or manipulate 3D objects or coordinate with other agents.; Among all the cues available for computing 3D information, the motion cue (also called the stereo, parallax or structure from motion cues) provides the most unequivocal information about the camera motion, calibration and 3D structure. Thus the study of the motion cue has been the subject of intense research in the computer vision community. The majority of the results have been established primarily within a Projective Geometric framework which is not easily exploited by the control and robotics community.; In the first part of this dissertation, we show how to further use a blend of novel techniques in Differential Geometry, Estimation Theory, and Optimization to improve our understanding of the basic geometric laws which govern the visual perception. This new perspective has initiated a series of new developments in and geometric insights to almost every classic problem associated to the motion cue, such as motion estimation, structure recovery and camera self-calibration. In the end, we are able to reach a coherent mathematical theory for multiview geometry. This theory also helps us to discover and analyze certain singularity, degeneracy and ambiguity inherent in the 2D to 3D reconstruction problem. Further more, the use of differential geometry allows us to extend the existing theory of multiview geometry to non-Euclidean spaces. The second part of this dissertation presents some initial attempts towards such a theory.; The proposed common mathematical framework between computer vision and control/robotics theory enables a better formulation of vision based control. In the third part of this dissertation, we will address two basic approaches to vision based control, namely visual servoing and visual sensing. These two approaches are demonstrated through two vision based control projects: vision based navigation of an unmanned ground vehicle and vision based landing of an unmanned aerial vehicle.