Research On Multi-source Target Fusion Tracking Method Based On Joint Histogram Representation

Target tracking is the basis of many video scene analysis and understanding tasks,such as visual monitoring,human-computer interaction,vehicle navigation and so on.A large number of tracking methods have been reported.These methods can be classified into single-spectrum tracking and multi-spectral tracking.Compared with the single-spectral target tracking system,the multi-spectral target tracking system has obvious advantages in terms of survivability,space-time coverage,reliability and so on.Therefore,it is widely studied,and the most representative is the fusion tracking of infrared and visible light.The infrared sensor is to detect the target radiation by the difference in thermal energy to form the image,making it better than visible light in poor lighting conditions,but it can't capture the color and texture features of the target.Visible light sensors can't perceive temperature,but when dealing with multiple thermal targets,it is usually better than infrared sensors,especially when there is a significant color and texture differences.Therefore,by combining their data,we can get better tracking performance than using a single sensor.Starting from the two aspects of target representation and target search,this paper proposes two fusion tracking algorithms of infrared and visible object based on joint histogram representation:1.The mean shift tracking algorithm of infrared-visible target based on joint histogram representation.Firstly,the histogram is used to represent the model,and the color histogram of the infrared image block and the color histogram of the visible image block in the given candidate state are calculated respectively.Secondly,the similarity between the color histogram of the infrared image block and its target template is calculated using the Bhattacharyya coefficient,the similarity between the color histogram of the visible image block and its target template is also calculated using the Bhattacharyya coefficient,and the two similarities are integrated into a novel objective function.Thirdly,we obtain its linear approximation of the function by performing multi-variables Taylor expansion on it,and we induce a target state transition formula from the current candidate state to the new candidate state by maximizing the linear approximation.Finally,based on the location-shift relation,the optimal target location can be recursively captured in the mean shift procedure.2.The particle filter tracking algorithm of infrared-visible target based on joint histogram representation.Firstly,the tracking result of the previous frame is the initial state,and the Gaussian random sampling particle set is generated by the six-parameter affine transformation model.Secondly,the color histogram of the infrared image block and the color histogram of the visible image block corresponding to the given sampling particle are calculated respectively,and the similarities between them and their corresponding target templates are calculated.Then,the weighted combination of the two similarities is taken as the observation likelihood function of the particle filter tracker,and the particle filter tracking program is started to obtain the posterior probability.Repeat the above steps for the remaining particles to get the posterior probabilities of all particles.Finally,the expectation of the products of all particles and their posterior probabilities is the final state of the target in the current frame.The algorithm can deal with the affine motion changes such as scaling,rotation and deformation of the target,and can also overcome the shortcomings of the kernel-based fusion tracking method which is easy to fall into the local optimum during the iteration.The experiment results show that the kernel-based fusion tracking algorithm has high real-time performance,and the fusion tracking method based on particle filter can be used to tackle with the affine motion of the target.The two fusion tracking methods perform well in dealing with occlusion,illumination changes and target intersection.