Multimodal Image Fusion And Tracking Algorithm Based On Performance Feedback

With the construction of "Safe City", intelligent video surveillance (IVS) systems based on visible light cameras start to be widely deployed around the corners of the city. All-weather, automatic, real-time IVS for important areas has achieved great attention by the countries all over the world. Thermal infrared cameras can achieve infrared radiation, and detect targets according to thermal radiation of the differences between targets and the surrounding environment. They have all-weather 24-hour working ability, and are beneficial supplement of visible light cameras.Visible and infrared image fusion and tracking algorithms in IVS systems are studied in this paper, including representative fusion based on static images and analytical fusion based on dynamic image in the image fusion. The main work is listed as follows:Firstly, Research background and significance of this paper is introduced, and current progress of image fusion and tracking algorithms at home and abroad in IVS systems are reviewed.Secondly, a regional-level feedback fusion algorithm for visible and infrared image based on performance evaluation is proposed to use object information in infrared image and background information in visible image fully. The proposed algorithm separates the images into the low and high frequency parts with NSCT transform first, and then adopts fractal feature to perform man-made object enhancement in infrared image. The area of object and background is obtained by threshold segmentation. In the design of low-frequency fusion rule, weighted fusion coefficients of object area and background area are selected as parameters, and genetic algorithm is used to optimize these parameters according to performance evaluation of image fusion results. The high-frequency part adopts regional weighted average fusion rule. Finally, Inverse NSCT transform is performed to obtain fused image using the optimized parameters. The experimental results demonstrate that, the proposed algorithm can combine object information in infrared image and background information in visible image, and the fused image has a stronger contrast, which can be beneficial for battlefield situation display and object recognition tasks.Thirdly, a visible and infrared image fusion tracking algorithm based on CoUpdate is proposed to solve the problem of lost track caused by complex environment. Based on the thought of treating visual tracking problem as "Center-Around" classification, the proposed algorithm extracts the pixels characteristics of the target and surrounding from visible and infrared image first, and then obtains tracking model by the Boosting algorithm. The confidence coefficient of pixels is calculated based on the classification results. Decision level fusion method is adopted to get likelihood image, and meanshift algorithm is used to estimate target position. Finally in the Co-Training framework the target tracking results are combined to CoUpdate the tracking model. The experimental results show that, the proposed algorithm can improve the robustness of tracking, and use multimodal image information effectively.Multimodal video monitoring equipment is used for acquring visible and infrared images, and further experiments are carried out to verify the proposed image fusion and the visual tracking algorithms. The comparative experiments of image fusion and visual tracking algorithms are done with the measured data after registrating these images in image preprocessing. The experimental results demonstate the effectiveness and applicability of the proposed image fusion and the fusion tracking algorithms based on performance evaluation.Finally, the main work and further research of the thesis is summarized.