Spatial integration and localization of dynamic sensors

Recent advances in local area networks and mobile/embedded computing technology allow for the availability of multiple sensors in typical environments. These sensors, which can include microphone arrays, video cameras, and motion detectors, are often used individually for object and sound localization in the initial phases of complex human-computer interaction applications such as speech recognition and automatic teleconferencing.; Individual sensors, however, often fail to localize objects or sound sources due to background noise, sudden changes in the acoustic and lighting conditions of the environment, and limitations of the available computing platforms. However, the integration of a distributed network of sensors provides a mechanism to achieve robust speaker localizations under the mentioned limitations.; The reliability of dynamic sensors often varies across different spatial locations. For example, a microphone array may be able to localize sound sources in its own vicinity but will usually fail for distant sound sources. The same is true for other object localization sensors such as video cameras, where object localization within obstructed spatial regions is practically impossible for the obstructed camera. Hence, the integration of multiple sensors must take the different reliabilities of the sensors into account.; This thesis allows for the different levels of access of a sensor to different spatial positions to be monitored by defining the spatial observability function (SOF). This definition allows sensors to be integrated using a Bayesian probability model while taking into account the validity of their results for different locations. Using this integration mechanism, it is experimentally shown that the accuracy of the sound source localizations for typical multiple microphone array environments is improved by an order of magnitude compared to traditional approaches at a signal-to-noise ratio of 0 dB.; One requirement of the integration of dynamic sensors is prior knowledge regarding their position and orientations. It is shown that the definition of the SOF results in a Maximum-Likelihood based sensor localization mechanism that allows the location and orientation of dynamic sensors to be estimated relative to one another. Experimental results using dynamic microphone arrays illustrate the ability to localize the arrays using only the acoustic information obtained from the environment.