Image Fusion Based On Feature Extraction

Image fusion is a kind of technology can merge relevant information from multi-images of the same scene into a single image.The integrated image will be more informative and more suitable for human visual perception.Nowadays,image fusion has been widely applied into military reconnaissance,medical diagnosis and remote sensing.In recent years,deep learning has become the most popular method in computer vision area.Deep learning method build neural networks by simulating the information processing mechanism of human brain,this strategy helps extract features from input data on different level,so deep learning methods have excellent ability on handling complicated data.SSAE(Stacked Sparse Auto-Encoder)is a kind of unsupervised deep learning method.The way SSAE trained doesn't need any labelled data.These characters are quite valuable in image fusion applications.Besides,SIFT(Scale invariant feature transform)is a kind of stable feature extraction method which has already widely applied in image matching applications.Considering the fact that,the selection of fusion rule widely determined by the divergence between two under fused images.Which makes SIFT will has a bright application prospect in image fusion area.Based on the research of traditional image fusion method,SSAE and SIFT,we introduced SSAE and SIFT into image fusion area,the main content of our research are as follows:(1)Based on the research of SSAE,we proposed a SSAE code based IR(infrared)object detection method.First,IR image is decomposed into small image blocks by using sliding window technology;Second,train an SSAE model by using small image blocks;Third,use SSAE model extract SSAE feature from each image blocks;Then,a non linear activation function called ReLu_tanh is proposed to convert SSAE feature to the IR object credibility(OR).Experimental results show that the proposed IR object credibility is able to detect the IR objects from IR images.Based on OR,we proposed a novel OR and NSCT(Nonsubsampled Contourlet Transform)based IR and visible image fusion method.First,images are decomposed into low frequency subbands and high frequency subbands by using NSCT;Second,low frequency subbands are decomposed into frequency subband blocks;Third,OR is adopted to evaluate the possibility of each frequency subband blocks has IR objects;Then,OR and ReLu_tanh are adopted to build a weighted based fusion strategy for the fusion of low frequency subbands;Next,the choose-max fusion strategy is adopted to fuse high frequency subbands;In the end the inverse NSCT is adopted to convert all frequency subbands into the fused image.(2)For practical image fusion applications,best fusion strategy can be chosen according to the divergence between two under fused images.We proposed a content compatibility metric for image fusion applications in this paper.First,find all SIFT descriptors in two under fused images;Second,pick those descriptors with short distance and same location as match descriptors;Third,divide the under fused images by using sliding window technology;Forth,calculate the number of match descriptors in each image block and build the content compatibility metric(cmatch)according to the number;So,image blocks with large cmatch value has more similar content.Based on cmatch,we proposed a cmatch and NSCT based image fusion method.First,source images are decomposed into low frequency subband coefficients and high frequency subband coefficients;Second,sliding window technology is adopted to divide low frequency subbands into subband coefficients blocks;Third,cmatch is adopted to evaluate the divergence between subband coefficients blocks;Forth,a combined fusion strategy is proposed to fuse each pair of subband coefficients blocks according to cmatch value.For the fusion of subband coefficients blocks with high cmatch value,weighted average fusion rule is more adaptable.For the fusion of subband coefficients blocks with low cmatch value,choose-max fusion rule can get better results;Next,all fused subband coefficients blocks are converted into fused low frequency subband coefficients by using inverse sliding window technology;Then,high frequency subband coefficients are fused under the choose-max fusion scheme;In the end,inverse NSCT is adopted to get fused image.Experimental results show that the proposed method is efficient in the fusion of source images with complicated contents.