Deep Learning Based Recaptured Screen Image Demoiréing

During digital image capturing,the captured images will suffer from an degradation called ”moiré pattern”,when there is aliasing between the object and the array of the camera sensor.Because of the grid pixel sampling in digital screens,it tends to produce moiré patterns when we capture the screens using digital cameras.We call this kind of moiré patterns as ”recaptured screen image moiré patterns”.The colors and shapes of moiré patterns are variable,which makes it difficult to separate them from the image content,and this degradation severely damages the visual quality of recaptured screen images.Nowadays,as recapturing screens has become a convenient and important way of recording information,it is of great significance to explore moiré removal algorithms for recaptured screen images.To our knowledge,there is only a few works for recaptured screen image demoiréing.Considering that deep learning has achieved excellent performance in image restoration,in this thesis,we construct two databases by synthesizing paired moiré free and moiré images and correspondingly propose two deep learning based moiré removal algorithms.The contributions of this thesis are summarized as follows:1.We propose a novel convolutional neural network(CNN)for recaptured screen image demoiréing.Since there is no explicit ground truth images for moiré images,firstly,we propose a moiré-free image synthesizing method based on image alignment and image decomposition.In order to reduce the difficulty of the mapping from the moiré image to the moiré-free image,the brightness of the synthesized moiré-free image is similar to that of the moiré image.The constructed Moir?e Pattern Image(MPI)database is the basis of supervised learning based demoiréing algorithm and image quality evaluation.Then,with the synthesized image pairs,we construct a CNN to learn the mapping from the screen moiré image to the moiré-free image.Finally,we train the network with mean square error loss,perceptual loss and adversarial loss.Our experiments demonstrate that the proposed method can remove the moiré patterns while preserving image details.The proposed method outperforms state-of-the-art methods in both objective and subjective measurements.2.We propose an Additive and Multiplicative Network(AMNet)for demoiréing and brightness improving.There is a significant brightness difference between the recaptured moiré image and the original screen content.The screen has high brightness,which is in line with human's visual preferences.Therefore,we propose a method for both demoiréing and brightness improving.Firstly,we synthesize the moiré-free image whose brightness is similar to that of the screen,and create a new database,called Moir?e Removal and Brightness Improving(MRBI)database.Then the AMNet is proposed to learn the the mapping from the low light moiré image to the high light moiré-free image.The AMNet consists of an addition module and a multiplication module,which jointly learn demoiréing and brightness improving.Our experiments show that the proposed algorithm can not only effectively remove moiré patterns,but also improve the image brightness,and outperforms state-of-the-art methods.