| Past research into multi-sensor data fusion have given rise to techniques that can be characterized as heuristic and ad hoc. In this thesis a formal framework for fusing different modalities of registered sensory inputs at the data level is introduced. The approach makes use of the calculus of variations, and the result is a mathematically rigorous method for improving estimation quality by simultaneously using all observed data as well as additional constraints. This variational fusion technique is demonstrated on the problem of three dimensional surface estimation from a set of registered images via the use of simulated range and intensity data, and illustrated through the use of real data. The technique is further demonstrated on the fusion of intensity with sparse range data. The results indicate that a four to five-fold increase in surface estimation accuracy with respect to the input data can be achieved. Furthermore, a 8% to 250% increase in accuracy over surface estimation from either sensing modality alone (i.e., via shape-from-shading or surface reconstruction) can be realized. The applicability of this fusion approach to other fusion problems is discussed, as are directions for future research. |
