Color Image Joint Demosaicking And Denoising Based On Convolutional Neural Network

Color image demosaicking and denoising are two important steps in the pipeline of the image signal processor(ISP)in digital cameras.In the traditional ISP pipeline,the above two tasks are usually considered independently.Therefore,the performance of one task might be affected by the other,which limits the accuracy of the final result.This thesis focuses on joint demosaicking and denoising(JDD)methods based on Convolutional Neural Network(CNN).For the input noisy Bayer image,the unified CNN network is designed to simultaneously complete the tasks of demosaicking and denoising,so as to finally reconstruct the clear full color image.Although applying deeper and wider CNN network can achieve better reconstruct quality,it is not practical to deploy large-scale network in many applications where the computation resource is highly limited.Therefore,this thesis proposes a lightweight CNN for color image demosaicking,and further expands it to a lightweight CNN for the JDD task.The contributions of this thesis are listed below:(1)A lightweight CNN for color image demosaicking network(LIDN)is proposed to achieve a better trade-off between the model performance and the parameters.Firstly,to effectively extract shallow features,a multi-core feature extraction(MCFE)module,which takes the Bayer sampling positions into consideration,is proposed.Secondly,by taking advantage of inter-channel correlation,an attention-aware fusion(AFF)module is presented to efficiently reconstruct the full color image.Moreover,a feature enhancement(FE)module,which contains several cascading aggregated enhancement blocks(AEBs),is designed to further refine the initial reconstructed image.To demonstrate the effectiveness of the proposed network,several state-of-the-art(SOTA)demosaicking methods are compared.Experimental results show that with the smallest number of parameters,the proposed network LIDN outperforms the other compared methods in terms of both peak signal-to-noise ratio(PSNR)and structural similarity index(SSIM).(2)Based on the above-mentioned network,a lightweight image joint demosaicking and denosing network(LIDDN)is further proposed.Firstly,a noise level features adaptive guidance(NLFAG)module is presented to learn the feature of noise levels in feature map at different stages.Secondly,a spatial and channel feature enhancement(SCFE)module with spatial attention and channel attention is designed to well separate the smooth and high frequency information,which leads to the improvement of the performance in the center of objects,at the edges or on the texture positions.Finally,an AFF+ module,which can be obtained by substituting the simple CA component in the ordinary AFF module with the AEB blocks,is used to enhance the change of information between channels.Experiments show that compared with other networks,LIDDN achieves the highest quality of the reconstructed full color images.