Human Tracking Based On Audio Visual Information Fusion And Its Application

This thesis mainly consists of two parts,one is the research about human trackingalgorithm based on audio and visual information fusion,the other is the design ofreal time spot archiving system。To overcome the weakness of traditional human tracking algorithms based onlyon visual or audio information,especially in complicated and dynamic environments,we fuse audio and visual information so that the tracking result becomes morereliable and robust。This thesis proposes a new human tracking algorithm based onaudio and visual information fusion。Firstly audio and visual information are usedseparately to track the target,then two prior distributions are proposed based on thetracking result。Particles are then sampled from these distributions and their weightsare calculated using the audio visual fused observation model。Then the postdistribution is calculated and the final tracking result is obtained。In our algorithmaudio and visual information are in a symmetric manner so that they can compensatefor each other to a better extent。We also introduce weights to audio and visualinformation to reflect their reliabilities。These weights are adjusted dynamically in thealgorithm。Experiments show that our algorithm is more robust than trackingalgorithm based only on visual information,and it is also robust to light change,background change,occlusion to some extent。Based on this algorithm a real-timehuman tracking system is implemented,which can be used in the smart classroom totrack the speaker and gather his voice。Real time spot archiving system is a hot research topic and a reasonable testenvironment in pervasive computing。We analyzed the basic application environmentand designed scenario,basic modules and system architecture for it。We also definesome important events in the system。The audio visual information fused humantracking system will be integrated in the real time meeting archive system,andspeaker identification and face recognition techniques will be further used to identifypeople,so that the meeting content can be analyzed。This thesis discusses our firststep results in this area。...