Acoustic Enhanced Video Tracking System Based On Smallaperture Mems Microphone Array

Target tracking technology has a wide application prospect in security monitoring,smart city,UAV reconnaissance and so on.Video target tracking based on image processing as one of the important fields has been a very high research heat.The traditional surveillance system based on camera is faced with an unavoidable problem that the monitoring in the blind angle of vision is completely ineffective.In order to solve this problem,this thesis presents a design scheme of an acoustic enhanced video tracking system which combines visual tracking with auditory tracking.The system has the ability to estimate the direction of the target sound source in real time and track the target according to the image sequence.By estimating the sound source that accompanies the target motion,the camera can turn to the target outside the field of view.This acoustically assisted camera operation mode is an important complement to the operation mode of traditional surveillance camera.This thesis firstly summarizes the theoretical basis of target detection and tracking based on video data and audio data.In the aspect of visual tracking,the video target detection method based on frame difference method and the video target tracking method based on Mean-Shift method are introduced.In the aspect of auditory tracking,the spatial sound field signal model received by microphone array is analyzed,and the acoustic target azimuth estimation algorithms based on microphone array are studied:CBF algorithm,MVDR algorithm and RCB algorithm.Then,the effects of SNR and sampling points on the performance of auditory localization methods are simulated,compared and discussed by simulation experiments.Considering the necessity of embedded system and the cost and size of the system in practical application,a small aperture array with 7 digital omnidirectional MEMS microphones is adopted in this thesis.The radius of the array is 4.5cm,which meets the miniaturization requirements of the system.Broadband beamforming and Mean-Shift algorithms are applied to auditory and visual data processing respectively.The acoustic localization results show that the average angle error is less than 3°,which provides an accurate initial field of view for the visual tracking module to locate the target outside the camera's field of view.Visual and auditory joint tracking experiment is carried out in a normal office environment,and the test results show that the tracking success rate of this system is 86.7%,and it meets the real-time requirements.