| This thesis sets out a Bayesian approach to the robust combination of measurements from multiple sensors in different measurement spaces. Classical least squares optimization is used inside a sequential Monte Carlo approach to find the most likely local estimate. The local optimization speeds up the system, while the Monte Carlo approach improves robustness in finding the globally optimal solution. Models are simultaneously fit to all the sensor data. A statistical approach is taken to determine when inputs are failing and should be ignored.; To demonstrate the overall approach described in this thesis, the 3D position and orientation of highly over-constrained models of deformable objects—fingers—are tracked. Accurate results are obtained by combining features of color and stereo range images. The multiple sources of information combined in this work include stereo range images, color segmentations, shape information and various constraints. The system is accurate and robust; it can continue to work even when one of the sources of information is completely failing. The system is practical in that it works in real time and can deal with complex moving backgrounds that have many edges, changing lighting, and other real world vision challenges. |
