Structure-aware Image Fusion

Image fusion is a technique of integrating complementary information in multiple source images obtained by multiple sensors in the same scene or target and fusing to generate composite images.Synthesizing the effective information contained in each source image,the fusion image is more comprehensive,accurate and stable than a single source image.One of the common image fusion methods,the fusion method within transformed domain,turns source images into the transformed-domain-coefficients,fuses the corresponding transform domain coefficients of each source image,and then inverses transformation to reconstruct the fusion image.The advantage of the widely used transformed-domain fusion is the effective capture of details in different scales,but a common issue of such methods is that the contrast and grayscale of the fusion image will deviate,and even affect the preservation of the original information in source images.In contrast,the fusion methods in spatial domain can directly process pixels in source images,thus preserving the details of the various scales without being limited by multiscale decomposition,while avoiding the computational burden of multi-scale transformation.The goal of fusion is not only to simply combine the complementary information in the source images,but also to achieve the fine integration of complementary features through certain fusion rules.Although many image fusion algorithms can perform well in objective evaluation index,they are not satisfactory in subjective vision due to poor connection of convergence at the edges between different features,which usually makes the fusion results appear unnatural,and even show the so called "block effect".To tackle the above problems,we propose a Structure-aware Image Fusion(SAIF)framework in spatial domain.First of all,a salient structure extraction(SSE)method is developed for the extraction of the large-scale structural contour information.In this module,in order to smooth the noise and eliminate the small-scale texture,the source images are pre-processed with improved adaptive local Wiener filtering.The forward difference method is applied to obtain the discrete gradient of the image,which is used as the saliency measure.Then,the salient structure is obtained by saliency measure comparison between original images and used as the initial fusing weight,which represents the pixels with the most prominent features in source images.Next,we analyze and summarize the characteristics and commonalities of two typical edge-preserving filters,namely,the guide filter and the bilateral filter,define a joint filter based on the analysis,and develop an iterative joint filter(Iterative Joint Filter,IJF)that enables structural retention and transfers the details in the neighborhood of the major structure in the guide image to the salient structure.The initial fusion weight obtained by the salient structure extraction step is optimized to match the edge characteristics of the source image more.The resulting visual effect of the transition of different areas of the fusion image is more natural and more in line with the human eye perception.Finally,the fusion results are constructed by combining the weight map and the source images.And we extend the algorithm to multiple source images fusion as well.The proposed SAIF method is more versatile than most of the existing image fusion algorithms and has a lower computational complexity than the state-of-art methods,solving the poor edge-linking problem in fusion images so that the visual effects are more natural.In order to verify the validity and superiority of the proposed method,we use three types of images for comparative experiments with five other state-of-art image fusion methods,and the qualitative and quantitative analyses are given from the subjective and objective evaluations.The fusion results show that the proposed SAIF framework has the best visual effects among the comparative fusion methods.It avoids the occurrence of halo phenomena,visual artifacts and pixel mutations,so that achieves more natural edge-linking in the result of image fusion.The proposed framework obtains the highest objective evaluation index among these algorithms as well,confirming the superiority and efficiency of it.