A Study On Multispectural Fusion Of Color And Near Infrared Images Based On Convolutional Neural Networks

Image fusion aims at fusing images taken by different sensors to generate a high quality target image,and have many applications such as video surveillance and autonomous cars.Multi-sensor fusion images not only contain the advantages of different source images,but also eliminate redundant information in the source images,and describe scenes more accu-rately and clearly.In recent years,the fusion of visible color and near infrared images has increasingly become one of the key research areas of image fusion.Compared with visible color images,near infrared images are not only less susceptible to external environments such as light and atmospheric conditions,but also have better details and contrast.A fusion image obtained by the fusion of the visible color image and the near infrared image is able to retain the color of the visible image with the details of the near-infrared image.With the continuous development of image fusion and artificial intelligence technology,deep learning plays an important role in image processing.Based on them,this thesis mainly investigates the fusion of color images and near infrared images based on deep learning.The main contributions of this thesis are as follows:1.We utilize a multispectral deep fusion network,which consists of three sub-networks:denoising,enhancing and fusion.In low-light conditions,the color image features are not obvious and highly degraded by severe noise.Thus,we use the denoising sub-network to eliminate the noise of the color image.After denoising,we use an enhancing sub-network to increase the brightness of low-light images.Because the near infrared image contains more details,we fuse the near infrared image and the luminance channel of color image through the fusion sub-network.This method can effectively fuse color images and near infrared images,and generate a high-quality fusion image containing textures and colors.2.We propose a color and near infrared fusion network for hidden texture recovery based on pyramid feature selection and attention map,i.e.a daylight fusion model.Compared with color images,near-infrared images are less susceptible to atmospheric environment and veg-etation characteristics.Therefore,we fuse color and near infrared images to recover hidden textures in visible light images.Since overexposure and underexposure cause dynamic range allocation problems,we use the attention map of gray image to guide contrast enhancement.Dark areas are more enhanced,while bright areas are less enhanced.In feature extraction,a pyramid feature selection module is used to extract multi-scale information to achieve better fusion result.In addition,we provide training data generation that the input smoothed color images are generated by their original color images,i.e.ground truth.Experimental re-sults show that the daylight fusion model successfully recovers hidden textures lost in color images while keeping colors3.Based on the daylight fusion model,we propose a low light fusion model of color and near infrared images.We synthesize a low light image dataset based on smoothing and gam-ma correction.We add the details of near infrared images into the smoothed color images to generate the ground truth.Low-light color images have more noise and less details,and thus we propose a denoising network to preprocess color images.We use concat operation in the denoising network to avoid the loss of image features during the learning process to obtain high-quality images.We independently train the denoising network because the denoising datasets are easy to obtain.Then,we use gamma correction to enhance the image after de-noising.Experimental results show that the low light fusion model generates high-quality images with little noise,fine details and good colors.