Research On Multi-Sensor Dynamic Image Sequence Fusion

In this thesis, we studied multi-sensor dynamic image sequence fusion. To realize the image fusion, we need to accurately preserve the useful information of the original images that contain dynamic scenes and appropriately fuse them into a single image. Then we expanded its dynamic range and preserved as much scene details as possible, and enhanced the contrast degree of the fusion image and removed the ghosting phenomenon caused by the dynamic target at the same time. The main work accomplished in this thesis is as follows:(1) We proposed the multi-sensor dynamic image deghosting method based on patch match. The proposed method finds the most similar between corresponding patches by using the patch match method,obtains the motion vector between similar patches, and completes the matching of each image patch. The experimental results showed that,compared with other contrast methods, this proposed multi-sensor dynamic image deghosting method based on patch match increased an average of 7.5%, 14% and 14.6% on the SSIM index, respectively, an average of 10.2%, 26.8% and 19% on the edge intensity index,respectively, and an average of 8.7%, 33.4% and 8.7% on the average gradient index, respectively.(2) We proposed multi-exposure dynamic image fusion method based on image alignment (MDIFA). The method adopted image alignment to align the input images with the selected reference image,and then fused the already aligned multi-exposure dynamic image sequence to obtain the final fusion image. Using MDIFA algorithm, we can achieve effective fusion performance for the multi-exposure image sequence. The fusion image kept the good exposed areas both in the lower exposure images and in the higher exposure images. The experimental results showed that, when compared with other contrast methods, the proposed method increased an average of 15% and 15% on the VIF index, respectively, an average of 10% and 10% on the QAB/F index, respectively, and an average of 14% and 14% on the FSIMc index,respectively.(3) We proposed multi-exposure dynamic image fusion based on PatchMatch and illumination estimation. The method firstly used the image alignment method based on PatchMatch method to achieve the alignment between the input images and the reference image, and then fused the aligned multi-exposure dynamic image sequence by using illumination estimation. At last, we obtained the final fusion image. The experimental results showed that, when compared with other contrast methods, the proposed method increased an average of 25%, 25%, 38%and 38% on the VIF index, respectively, an average of 37%, 39%, 57%and 59% on the QAB/F index, respectively, and an average of 10%, 10%, 9%and 13% on the FSIMc index, respectively.(4) Based on these three proposed methods, we developed and accomplished a multi-sensor image sequence fusion system in this thesis.The system realized the visual representation of these three methods, and added the image quality assessment module for these fusion methods proposed in this thesis, to verify the effectiveness of the proposed methods.