Research On Deep Neural Network Image Inpainting Model Guided By Structural Information

Image inpainting is a technology that uses effective algorithms to restore the missing areas in an image.Image inpainting is used in many fields in daily life,such as medical,military,art,education and so forth.Therefore,it has received extensive attention from academia and industry,and has become one of the research hotspots in image processing.Image inpainting repair the damaged image by repairing the structure information and color texture detail information of the missing area in?the image.Existing image inpainting methods based on deep learning have significantly improved the effect.However,these algorithms usually reconstruct blurred and uneven color transitions for missing areas of the image.In response to the above problems,this research has mainly done works as follows:(1)An edge information and mask shrinking based image inpainting method is proposed to constrain the structural information and color transition information of the damaged area simultaneously.This research first repairs the edge information of the damaged area of the image in the first stage,thereby constraining and repairing the structural information of the missing area.Secondly,this research uses the edge information repaired in the first stage as guidance to guide the image inpainting in the second stage.This method simultaneously consider and repair the structural information and color information of the missing area.(2)In the first stage,the binarized edge map of the image is repaired.This paper reconstructs the structural information of the missing area based on the generative adversarial network.In generator,due to the lack of effective information in the missing area,this research did not add a skip connections between the encoder and the decoder to transmit the underlying information.Besides,the generator uses residual networks to solve the problem of vanishing gradients in deep neural networks.This research repairs the color transition information of the missing area in the second stage.Since the binarized edge information completed in the first stage will be used as a guidance to enter the network in the second stage,the generator of the second stage uses skip connections to provide the decoder with more primitive underlying structure information.The residual network is also used in the generator of the second stage.(3)The mask shrinking strategy is applied to the second stage model to repair the color and texture details of the image.In the second stage,this research uses the mask shrinking strategy to repair the color and texture details of the missing area under the guidance of complete binarized edge information.This strategy introduces partial convolution to reduce the noise caused by the convolution operation in the image inpainting process.At the same time,the mask update mechanism is added to the network encoding and decoding process of the image to record and track the image inpainting process.(4)The structure information of the image has been improved.this research replace the binarized edge map with an edge gradient map to represent the structural information for image.On the one hand,the edge gradient map carries more effective structural information,which provides more available information for the neural network to repair the structure of the missing area.On the other hand,there are transition between the edge area and other area in the edge gradient map relative to the binarized edge map,which is friendly to the neural network and makes it easier for the neural network to repair the structure of the missing area.The experimental results show that the two-stage image inpainting method based on generative adversarial network proposed in this paper has outstanding performance in the inpainting effect,compared with other state-of-the-art image inpainting algorithms.And the guiding effect of the edge gradient map is better than that of the binarized edge map.