Generative Model In Structural-Hankel Domain For Image Inpainting

Image inpainting refers to the repair of missing,damaged,or contaminated regions in an image by means of algorithms and mathematical models.In recent years,research in this field has been very active,and the restored images obtained by traditional image inpainting methods dealing with damaged regions are often limited in semantic and texture structure consistency.The rapid development of deep learning provides more options to solve image inpainting challenges.However,existing deep learning methods based on generative models generally suffer from inflexible model structures,difficult training and lack of robustness,and the training process requires thousands of high-quality datasets,which consumes a lot of time and cost.In this study,we propose a novel idea to construct low-rank structured-Hankel matrices from only ten or even fewer image samples to assist score-based generative models for color image inpainting tasks.The main contributions are as follows:(1)Unlike traditional generative models that require complex models to accommodate multiple sample data distributions,a score-based generative model through stochastic differential equations can smoothly transform complex data distributions to known prior distributions by gradually adding noise.The inverse process of the model then subtly converts the prior distribution back to the original data distribution by gradually eliminating noise.This study explores the intrinsic structure of the model and uncovers a general prior information of image structure,which can be introduced into the iterative inpainting process to restore images under different mask damages,effectively solving the problem that the model needs to be trained repeatedly when processing different samples.(2)To address the common problem of existing studies that a priori information learning requires high quality and large datasets for long time training,this study models the tensor from the structural-Hankel matrix.First,the internal intermediate patches are randomly extracted from the images,and then these image patches are constructed into structural-Hankel matrices.In order to better apply the generative model to learn the internal statistical distribution of the image patches,the large-size Hankel matrix is collapsed into a high-dimensional tensor for efficient training of the model.In the iterative inpainting process,the inpainting problem is considered as a conditional generation process in a low-rank environment,and the missing information is inferred based on the gradient of the corresponding data distribution in the structural-Hankel domain estimated by the score matching network,combined with stochastic differential equation solver,alternating direction method of multipliers and the data consistency term step.Extensive experimental results show that this research method can recover high quality images through fewer training samples,with stable model training and better generalization ability,and has high application value in the field of image inpainting.