Algorithm Of Audio And Visual Fusion For Localization And Tracking Based On Audio Auxiliary Information

The localization and tracking of moving objects is an important research direction of indoor intelligent system scene analysis.The traditional location tracking method mainly uses a single audio and video information to track the target.However,both video tracking and audio tracking have their own limitations in complex scenarios.In order to make the tracking system performance of moving objects better,this paper discusses a new algorithm of audio-video information fusion based on particle filter.In the audio positioning section,we mainly discuss the algorithm of TDOA-based sound source localization,and introduce the generalized cross-correlation(GCC)method of PHAT weighting.In order to improve the positioning accuracy,the PHAT-GCC method is improved by multi-frame weighted smoothing.Experiments show that the improved algorithm has good performance in anti-noise and anti-reverberation.In addition,the estimation of source position under time delay estimation is introduced,the calculation of sound source position is deduced and the inherent error of the spatial structure is analyzed.In the video processing part,Mean Shift algorithm is used to track the target,and the color feature of the target region is extracted and the mean shift is processed.Iterative search to the candidate template and the target template most similar location is the target location,and then achieve tracking.In the occlusion problem,a new template updating strategy is used to improve the original algorithm.The experimental results show that the template update strategy effectively prevents the occlusion of the target template,and solves the problem of occlusion.In the information fusion part,this paper proposes a method of image information fusion based on feature validity in the framework of particle filter,and combines the TDOA feature and color feature in the observing steps to realize the complementarity principle of heterogeneous information.The mean drift method is embedded in the particle filter to improve the real-time performance of the tracking system.Finally,the template updating strategy is used to ensure the correctness of the target template in occlusion.The experimental results show that the improved tracking algorithm has better performance than a single tracking algorithm.