Mosaic Image Restoration And Recognition Research Based On Deep Learning

With the rapid development of artificial intelligence and computer vision,image processing based on deep learning has brought on development booms.As a kind of neural network,convolution neural network can compute directly with image pixels through convolution kernel,and extract image features from image information level by level.This process comes closer to the visual information processing mechanism of human brain,and also has a good effect in application and research.However,convolution neural network,as a data-driven model,needs a large number of color images sets in the training process,and the color images are obtained by the image restoration from Mosaic images.In order to improve the image quality of color images,the Mosaic image restoration methods based on deep learning also achieve good results.However,in practice,for most digital cameras the Mosaic image is captured by a CCD/CMOS image sensor with a color filter array covering the surface of the pixel array.And the pixel point on the sensor receives information about the intensity of the light,in other words,each pixel covered by a CFA can only be sensitive to one color.The arrangement of the color sensitive pixels determines the data format of the Mosaic image.The Bayer color filter array is widely used,and most of the color images are obtained by Bayer CFA Mosaic images through the image restoration algorithm.However,the convolution operation of the existing deep learning methods is designed for RGB color images,and not adapted to Mosaic images because of the special arrangement.Therefore,in this paper we design the phase-convolution operation for the periodic arrangement of Mosaic images pixels to extract the features of the Mosaic image,and then a demosaicing image restoration method based on Generative Adversarial Networks and phase-convolution is designed.In addition,in many end-to-end applications such as long-term exploration,reconnaissance and surveillance,people only focus on the results of image classification,recognition and understanding and the restoration of color images is more intended to satisfy the visual requirements of the human eye.In this case,it is enough to train and test networks on mosaic images without the process of color demosaicking.This simplification can avert complex algorithms,decrease the computation burden and reduce power consumption,which is particularly important for mobile devices.So,we propose a deep classification network called Mosaic Net to classify and recognize Mosaic images.The Experiments show that the proposed Mosaic Net model has better classification accuracy than the standard classification network model with color image.Finally,we establish a Mosaic image dataset,train the Mosaic Net model with our dataset,and verify the feasibility of the deep learning algorithm directly with Mosaic images.