Deep Learning Based Recaptured Screen Image Demoiréing

When the electronic screen is photographed by a mobile phone or camera,moiré pattern will inevitably appear in the captured image,seriously reducing the quality of the image.These moiré artifacts are caused by the slight discrepancy of spatial frequency between the camera sensor array and the sub-pixel layout of LCD monitor.Due to the different shapes,varying colors and spanning a wide range of frequency bands of moiré pattern,image demoiréing becomes a hotspot and difficulty in the field of image restoration.In practice,removing moiré pattern by image processing software requires high professional skills,while traditional filtering methods have poor effect of inhibitory on moiré pattern.Recently,with the development of deep learning approaches,convolutional neural networks have achieved excellent performance in image restoration task.Based on the merits of deep learning,this paper proposes two image demoiréing methods to effectively remove the moiré pattern from screen images.The main contributions of this paper can be summarized as follows:(1)This paper proposes a dual-domain distilling network for image demoiréing.As the moiré pattern spans a wide range of frequency bands which makes image demoiréing challenging,the proposed method designs a dual-domain network for demoiréing,which simultaneously considers the spatial-domain and frequency-domain priors for precisely removing moiré pattern from a single image.Inspired by distillation mechanism,the proposed model introduces a process-oriented learning strategy to guide the process of moiré pattern removal,with a process-oriented loss designed for measuring the similarity of features between teacher and student networks.Extensive experiments are carried out to show the effectiveness of the proposed method compared with other competitors.(2)This paper proposes a generative adversarial network based unsupervised method for image demoiréing.Existing image demoiréing methods are most supervised,which heavily rely on paired images that are expensive to collect in real-world applications.To mitigate this problem,a novel unsupervised image demoiréing is proposed,which adopts a generative adversarial network as the backbone.Specifically,the proposed method explores a color-based prior for predicting coarse location of moiré pattern.With the help of the captured position map guided by the prior,an effective moiré pattern extractor is then designed to accurately extract moiré pattern masks.Besides,a background loss is designed to simultaneously suppress the residue of moiré pattern left in the recovered images and preserve the details of clear images as much as possible.Extensive experiments conducted on public datasets demonstrate the effectiveness of the proposed unsupervised method.