Design And Implementation Of Moving Target Tracking System Based On Video And Audio Information Fusion

With the progress of science and technology, mobile robot technology is developing rapidly, and is applied to the human life gradually. How to make the robot to track the target effectively and efficiently has important research significance and high application value. Most of traditional target-tracking algorithm rely on the single sensor. So the tracking accuracy will decrease while it is influenced by some environmental factors. In order to improve the performance of tracking algorithm, a target-tracking system based on the fusion of video and audio information with the Kinect sensor as the information acquisition device is designed and implemented in this dissertation.Firstly, at the visual level, the method of particle filter based on the characteristic value of HSV color histogram is used to realize the video object tracking in this system. The method has good real-time performance, and can be used in dynamic background. In the course of study, it is found that the method can solve the problem of occlusion. But when the target is out of the video area and the condition of illumination changes, the method is less robust.Secondly, at the auditory level, the sound source localization method based on delay estimation is researched and realized by combining the microphone array structure of Kinect. This method has the advantages of real-time and high accuracy in the ideal environment. But it is only for a single source, without the ability to identify the type of sound source.Finally, according to the stable using environment of the system and the target will not continue to send out sounds, an information fusion decision is proposed, which is based on video information as the main source and sound information as the secondary source. In the decision-making stage, the initial tracking and positioning results are effectively integrated to achieve the goal of tracking. Confidence criterion is adopted to evaluate the results of visual target tracking. If the results are not reliable, then use the audio positioning to correct the tracking result.Test results show that the video and audio fusion tracking method achieves an improved accuracy compared with the target tracking algorithm based on single sensor. In some special environments, such as illumination, occlusion and object, the method has good robustness. For subsequent research and application, due to the relative independence between the modules of the system, just provide the robot control module, the system can be easily transplanted to the robot platform.