Research On Image Diversity Inpainting Based On Generative Networks And Text Guidance

Image inpainting is the restoration of damaged images using computer related methods.To address the shortcomings of existing restoration algorithms,two image restoration algorithms are proposed,and the main research work of the thesis is as follows.To solve the problems of unreasonable structure,and low fineness for details in existing algorithms when the irregular damage area is too large and the background details are complex,a dense multi-scale fusion cavity convolution-based image restoration algorithm is proposed.Firstly,the damaged image samples are fed into a global structure generation network,whose the core module is a dense multiscale fused cavity convolution block.Secondly,the output samples of the structure generation network are fed into a detail generation network,which contains a dense multi-scale fused cavity convolution block and parallel self-attentive layers.Finally,the output samples are enhanced with a spectral discriminator for global and local content consistency and detail features.Experiments are conducted to compare the proposed algorithm with the state-of-the-art in recent years on the international public datasets.The experimental results show that the proposed algorithm can achieve restoration of images with excessive broken areas,and the restoration has smooth boundaries and clear details to satisfy visual coherence and realism.The proposed algorithm outperforms the comparison algorithm in both subjective and objective evaluation indexes of the restoration effect.In order to solve the problem that the current repair method has poor effectiveness of the repair results and the repair process is unguided under the condition of large-area damage to the image,an image diversification repair algorithm guided by the label text is proposed.The controllability and diversity of repair results.First,the semantic content of damaged locations is extracted by designing dual multi-modal mask attention;then,a deep text-image fusion module is introduced to enhance the fusion degree of multi-modal data,and the semantic content between generated samples and labels is maximized by applying matching loss information relevance;finally,the authenticity of the generated samples is enhanced with a projective discriminator.The workflow of the proposed algorithm takes the text label as the guide,and repairs the specified image according to the successfully matched label words.In the comparison experiment,the output sample effect is better than the comparison algorithm,and the generative diversity of the algorithm is proved in the controllable diversity experiment.