Research On Text-guided Image Inpainting

Image inpainting is the task of reconstructing the missing region,and it is a challenging topic in the field of computer science.This technology originated from the restoration of damaged handicrafts by restorers,and is now widely used in the field of cultural life,bringing great convenience to people.With the rapid development of deep learning,significant progress has been made in the field of image inpainting.In recent years,image inpainting technology based on deep learning has been widely used,which greatly improves the performance of image inpainting.However,when the missing region is large,most methods may encounter the problem of not being able to make accurate semantic inference due to insufficient contextual information.To further improve the quality of image inpainting,the following work has been done in this thesis:(1)Filling an image with information from the image itself alone does not achieve the filled content required by the user,while text descriptions can generate visually relevant semantic content,in order to better fill in missing areas in the image,this thesis proposes a text guided image repair model based on the dual attention mechanism,By introducing text description information into the image repair process,it provides context information for image repair and guides the repair of missing areas.The framework includes dual attention modules for spatial and channel information to obtain information related to repair tasks,and designs a comparative learning module to solve the problem of repair ambiguity;In addition,the framework also introduces text image matching losses to improve the similarity between the repaired image and the original image.(2)In recent years,generating confrontation networks has achieved good repair results in image repair,while GAN networks have played an important role in repairing images.Therefore,combining GAN and text information to generate image information can not only compensate for the shortcomings of GAN in generating images,but also provide guidance for image repair content using text information.However,the original generated confrontation network has the problem of gradient disappearance,and the quality of generated images is unstable.In order to improve the quality of image restoration,a text guided image restoration model based on generating confrontation networks was proposed.In order to stabilize the training process and improve the image repair effect,the model uses the least square method to generate an adversarial network,and further stabilizes the repair effect by introducing perceptual loss.Comparative experiments on CUB-200-2011 and Oxford 102 datasets show that the method proposed in this thesis outperforms the compared methods in terms of relevant indicators,and that text guided image repair has practical significance.