Research On Color Image Joint Demosaicking And Super-resolution

As two practical and important image processing tasks,color image demosaicking(CDM)and super-resolution(SR)have been studied for decades.In consumer cameras,due to the limitation of Complementary Metal Oxide Semiconductor(CMOS)sensor,there is only one color channel information for each pixel in the sample image.Therefore,the CDM algorithm is required to restore the full-color images.To faithfully reconstruct the high-resolution(HR)images from the given low-resolution(LR)images,the SR technology is widely used in many applications,such as video monitoring,medical imaging and so on.However,most literature studies these two tasks independently,ignoring the potential benefits of a joint demosaicking and super-resolution technology(JDSR).Therefore,in this thesis,a convolutional neural network(CNN)-based CDM algorithm is first proposed,and then a CNN-based JDSR algorithm is developed.(1)Most of the existing CNN-based CDM algorithms cannot achieve satisfactory results with a relatively low computational burden and model size.To deal with this problem,a color image demosaicking network based on inter-channel correlation and enhanced information distillation(ICEID)is proposed.Firstly,to fully utilize the inter-channel correlation,a guided-reconstruction structure is designed to obtain the initial CDM result.Secondly,an enhanced information distillation mechanism,which can efficiently extract and refine features from images,is presented to enhance the CDM result.Experimental results demonstrate that compared with many mainstream CDM methods,the proposed method is able to achieve significant improvement in terms of both objective and subjective quality.Meanwhile,the proposed method has a relatively low computational burden and model size.(2)A well-designed two-stage convolutional neural network(TSCNN)architecture is proposed for JDSR task.For the first stage,by making use of the sampling pattern information,a pattern-aware feature extraction(PFE)module is presented to extract features directly from the low-resolution Bayer-sampled image,while keeping the resolution of the extracted features the same as the input.For the second stage,a dual-branch feature refinement(DFR)module effectively decomposes the features into two components with different spatial frequencies,on which different learning strategies are applied.On each branch of the DFR module,the feature refinement unit,namely densely-connected dual-path enhancement blocks(DDEB),establishes the sophisticated nonlinear mapping from the LR space to the high-resolution space.To achieve a strong representational power,two paths of transformations and the channel attention mechanism are adopted in the building block of DDEB.Extensive experiments demonstrate that the proposed method is superior to the sequential combination of state-of-the-art(SOTA)CDM and SR methods.Moreover,with a much smaller model size,TSCNN also surpasses other SOTA JDSR methods.