Research On Multi-scale Encoding And Structural Priori Information Guided Image Inpainting Methods

Image inpainting refers to inferring and reconstructing the structure and texture of the missing regions according to the information of the undamaged regions in the input image and obtaining semantically sound and visually realistic results.Image inpainting has wide application prospects in image editing,medical and health care,and heritage conservation.With the continuous improvement of computer performance,learning-based image inpainting technology has made up for the shortcomings of traditional inpainting methods in image semantic understanding,and has made remarkable achievements,and become the research focus of many scholars at home and abroad.In this thesis,multi-scale features and appearance flow or attention feature propagation mechanism are combined under the framework of generative adversarial networks(GAN),which effectively solve the problems of artifacts,local color difference,structural distortion,and detail blurring when inpainting large missing areas.The main research contents and contributions are summarized as follows:1.The development of existing image inpainting methods was summarized and sorted out,and the advantages and disadvantages of mechanisms of different types of inpainting methods were explored,which provided the research basis for subsequent chapters.The inpainting mechanism and experimental results of CA,EC,and SF are analyzed in depth.It is found that the existing image inpainting methods have some problems such as artifacts,local color differences,structural distortion,and detail blurring when inpainting large missing areas.2.The traditional codec inpainting architecture uses a single-stage encoder,which leads to the gradual loss of shallow texture features along with the deepen of the network,resulting in artifacts and local chromatic aberrations in the inpainting of large missing areas.Considering the different feature information contained in different level features of the same scale,an image restoration model based on a multi-stage encoding network(MSEN)is proposed.MSEN uses a multi-stage encoder to compress and encode broken images of different scales and upload encoded features step by step to achieve the fusion of deep and shallow features.In addition,MSEN uses a skipping connection to reuse the multi-scale features obtained from multi-level encoding network encoding to the decoding stage,which further assists the decoder to generate fine textures.Qualitative and quantitative experiments show that MSEN effectively solves the problem of artifacts and local chromatic aberrations when inpainting large missing areas.3.The existing cascaded inpainting network uses unreasonable structural prior when inpainting large area damaged images,resulting in structural distortion and fuzzy details.This thesis introduces appearance flow and attention feature propagation mechanism and proposes an appearance flow based structure prior guided Image Inpainting(AFSP)methods.Firstly,to recover reasonable structural prior,the structure generator is regarded edge-preserved smooth images as the global structure of the image meanwhile uses appearance flow to sample features from missing regions.After obtaining reasonable structure priors,the texture generator synthesizes high-frequency details using contextual attention.Experimental results show that the proposed model is superior to the existing mainstream baseline methods in terms of global structure and local texture.4.The current image inpainting algorithms are lack software that can realize human-computer interaction functions,which is not conducive to widespread application in engineering practice.An image inpainting application software is designed and developed which combines the image inpainting algorithm proposed in Chapter 3 with the technology of developing a front page based on Python+Pyqt5.Finally,the image inpainting system is verified by the system implementation that it can be applied to the actual scene.