A Study On Light Field Image Super-resolution,Multiexposure And Multispectral Image Fusion Using Convolutional Neural Networks

With the continuous improvement of image requirements,image capturing equipment has been greatly developed and upgraded.Camera resolution is higher and higher,and the variety of imaging devices also appear.However,in some cases,hardware devices can not fully meet people's needs.Image processing technology can improve visual quality of the captured images.Light field camera is a new type of camera in recent years,which receive more and more attention.It has many advantages that ordinary cameras do not have,but its resolution limits its application.In this paper,the method of virtual angle synthesis is studied to improve the its angular resolution,which can improve the the resolution of the light field camera.Image fusion can fuse multiple images into an single image with more information,which has been widely used in many fields.In this paper,the multi-focus image fusion is investigated to solve the problem of limited single focus area of camera.Near infrared camera has attracted more and more attention in recent years.It can obtain high-quality images in nonideal environments but lack of color information.This paper studies the fusion of near infrared image and color image to restore the hidden details and texture of the color image.The main works and contributions of this thesis are as follows:1.A high-quality virtual view generation algorithm for light field camera based on densely connected convolution neural network(CNN)with multi-loss is proposed.With the emergence of commercial light field cameras,light field images have become more and more common.However,due to limitation of hardware,there is a balance between angular resolution and spatial resolution in light field cameras.Therefore,these cameras usually have sparse sampling in the spatial domain or angular domain.In this paper,we use densely connected CNN to alleviate this problem.Specifically,we propose a new kind of multi loss function,which trains the convolutional neural network model by minimizing the pixel loss,feature loss and edge loss of the generated perspective image and the ground truth image.The experimental results show that the proposed method can synthesize high quality virtual views,which is better than other methods.2.We present multi-focus image fusion method using light field data and CNN.We use light field data to generate focus maps for image fusion using refocus images.Unlike other CNNbased fusion methods which treat multi-focus fusion as binary classification to generate focus maps for fusion,we directly generate focus maps in fully convolutional networks via the multi-focus images.To train CNN directly,we construct a multi-focus image dataset which contains both multi-focus images and their ground truth using light field data,and utilize it to train the proposed networks.Experimental results show that the proposed method generates accurate focus maps close to the ground truth as well as outperforms state-of-theart fusion methods in terms of quantitative measurements.3.We propose to use a near infrared(NIR)image to recover the texture and detail of its corresponding color image for the same scene.We provide a method to fusion the NIR and visual image for same scene,which can integration their advantages together to get high quality fusion image.We divide the fusion task to two part: luminance and chrominance.We use the CNNs to fuse the NIR image and the luminance component of color image.The proposed method achieve an outstanding performance in texture,detail and structure in lowlight,high-light,haze and other conditions.We use CNN to reconstruct the chrominance through generated luminance image and original RGB image.Experimental results show the superiority of the proposed method over state-of-the-art methods.