Research On Multi-source Image Fusion Algorithm Based On Multi-scale Transformation And Neural Network

With the continuous development of imaging sensors,the amount of information contained in images becomes more and more abundant.The images presented by different types of sensors are quite different,and the images generated by the same type of sensor under different parameters or different conditions are also different.Multi-source image fusion technology synthesizes images acquired by different types of sensors or the same type of sensors under different parameter settings,so as to make the information of the images more comprehensive and richer,making up for the limitations of single type of image,and retaining the characteristic information of source image.Multi-source image fusion technology has high application value in several fields,such as AI medical,digital imaging,and video surveillance.In this paper,the fusion of multi-modal medical images and multi-focus images has been studied in depth from two aspects.On the one hand,a method based on frequency transformation is used to decompose the image in multiple scales.On the other hand,the neural network method is used to extract features of the image.The specific content is as follows:1.Medical image fusion algorithm based on heuristic search PCNNThe problem that the parameters of Pulse Coupled Neural Network(PCNN)are difficult to set and the detailed information is ignored when processing the low frequency sub-bands is addressed.In this paper,a PCNN based on Improved Quantum-behaved Particle Swarm Optimization(IQPSO),named IQPSO-PCNN,is proposed as a medical image fusion algorithm combined with Nonsubsampled shearlet transform(NSST).First,the NSST multi-scale decomposer is used to decompose the multi-source medical image to obtain low-frequency and highfrequency sub-images respectively.Then,for low-frequency subbands,the fusion rules of two different functions are presented,which simultaneously addresses two key issues of energy preservation(EP)and detail extraction(DE).For high-frequency subbands,unlike conventional PCNN-based methods,parameters are manually set based on experience,and the decomposed high-frequency subbands share a set of parameters.The IQPSO-PCNN model can obtain the optimal parameters for each high-frequency subband adaptively according to its own information.Finally,the fused low-frequency subband and high-frequency subbands are inversely transformed by NSST to acquire the final fused image.The experimental results show that the algorithm not only eliminates the artifacts on the fusion boundary,but also more clearly shows the important areas in the brain tissue.2.Medical image fusion algorithm based on edge-preserving PCNN and improved sparse representationTo address the problems that the sparse representation algorithm has redundant detail information when dealing with low frequency subbands and that PCNN ignores boundary information of different modes when dealing with high frequency subbands.This paper proposes a novel multi-modality medical image fusion method based on gray medical images and color medical images.In the proposed method,the NSST is used to decompose multi-source medical images into low-frequency sub-bands and several high-frequency sub-bands,taking advantage of the NSST's feature of multiscale and multi-directional decomposition of images.The improved sparse representation is utilized to fuse the low frequency sub-band,which can remove the detail features through the sobel operator and the guided filter to improve the ability to preserve energy effectively.Meanwhile,the high frequency sub-bands are fused by a pulse coupled neural network(PCNN)based on edge preservation.This method fully considers the imaging characteristics of different medical modalities,which can process edge information well and process image details better.Finally,the fused low frequency sub-band and high frequency sub-bands are inversely transformed to obtain the final fused image.Experimental results show that the fusion method based on edge preservation not only improves the fusion efficiency of image edges,but also the order of fusion does not influence the fusion effect.3.Multi-focus image fusion algorithm based on two-stage convolutional neural networkThe Convolutional Neural Network(CNN)based fusion method is difficult to obtain the accurate decision map and the fused boundary has some problems such as artifacts.In this paper,a two-stage convolutional neural network fusion method is proposed.In the first stage,an improved Dense Net is trained to classify whether the image patch is in focus or defocus,and then the corresponding fusion rule is utilized to acquire a perfect decision map.In addition,a multi-version blurred dataset is designed to improve the generalization ability of the network.In the second stage,edge-deblurring generative adversarial networks(EDGAN)is introduced to process the boundary.Furthermore,five different loss functions are applied to generate approving boundary deblurred images.At the same time,natural images are selected from the COCO dataset for special processing to simulate the boundary blurring situation to create the second-stage dataset.After two stages of processing,an image with rich details and decent fusion boundaries are attained.Experimental results show that the algorithm can obtain accurate decision maps by using the proposed first-stage framework,and the second-stage can eliminate the artifacts of the fusion boundary.