Vision-based control and coordination of unmanned vehicles

In this dissertation; we address the problem of using computer vision as a sensor for control and coordination of intelligent unmanned vehicles. We consider two challenge problems as motivating applications, and we develop vision-based motion estimation algorithms and control strategies to meet these challenges.; Our first challenge problem is vision-based landing of an Unmanned Air Vehicle (UAV). Here, we use a camera on-board a UAV to perform motion estimation and to control its landing onto a possibly moving target. We present a robust multiple-view motion estimation algorithm and the implementation of a real-time vision system based on commercial off the shelf hardware. Through flight test results with our rotorcraft UAV testbed, we demonstrate a motion estimation accuracy of within 5cm in translation and 5° in orientation. Further, by placing the vision system in the control loop of our UAV, we demonstrate vision-based landing onto a stationary target and tracking of a moving target which simulates ship deck motion.; Our second challenge problem is that of omnidirectional vision based leader-follower formation control for Unmanned Ground Vehicles (UGV). We take a novel approach where we consider only visual communication between vehicles, and specify the desired formation in the image plane of each vehicle. We contribute several novel motion segmentation and estimation algorithms for central panoramic cameras. We further present a distributed vision-based formation controller that ensures leader-to-formation stability while naturally incorporating collision avoidance by exploiting the geometry of omnidirectional vision.; Our overall research thrust has been to develop robust and real-time vision algorithms for motion estimation, control and coordination of unmanned vehicles. We have developed algorithms, implemented real-time vision systems, and integrated them into the control loops of our fleet of UAVs and UGVs. By demonstrating vision-based control systems in challenging outdoor environments, we have taken a step toward the future of vision-based intelligent unmanned vehicles.