Image Colorization Methods Based On Image Disentanglement

With the development of computer vision technology,many techniques related to image vision have applied to many scenes and fields in recent years,such as identity authentication,security,medical treatment,shopping and so on.However,high-performance computer vision algorithms often take the deep learning network as base model.The end-to-end deep learning models are difficult to extract semantics which human beings are familiar.This makes it difficult for users to understand the context of the results.For computers,it is also difficult to process human instructions directly.In computer vision tasks,there are still problems that the model lacks interpretability and there is a semantic gap between the model and high-level semantics.In computer vision tasks,the color of an image is very important for conveying information and expressing emotion.Color could affect people's cognition,behavior and decision-making.At the same time,the color of the image could affect the performance of computer vision models.However,due to imaging equipment and lighting,many precious visual materials are in black and white mode,or the color effect of images is not satisfactory.Therefore,colorizing a grayscale image,repairing or editing the color of an image have very important historical significance and commercial value.In order to improve the lack of interpretability in the image colorization task and the "semantic gap" with human instructions,this thesis considers using disentangled representation learning methods to achieve the image colorization task.The disentangled representation learning methods can separate the interpretable attributes in the data and generate interpretable disentangled representations,so that people can understand the meaning of model's feature map.For the model,the interpretable representations not only increase the trustworthiness of computer-aided decision-making,but also improve the robustness and generalization ability of the model.Aiming at the problems of the lack of interpretability of the image colorization models and the semantic gap with high-level semantics,this thesis proposes a text-guided image colorization method based on image disentanglement.This method associates the semantic information of the text with the color,and obtains the disentangled representations of the color information and content information of the image by learning the two-way mapping between the image and the latent representation.Then,the semantic information of input text is mapped to the space where the disentangled color representations at.This method can generate colorization results that conform to the text semantics by combining the semantic color representation from the text and the content disentanglement representation of the target image.The experiment is implements on two public datasets.Through a large number of quantitative and qualitative evaluations,we prove that this method can effectively disentangle color information and content information of the images,and achieve better colorization effects than the baseline method.In order to solve the problem that the reduction of colorization performance by the mutual influence between the objects and the background when the image contents are complex,this thesis proposes an instance-aware colorization method based on image disentanglement.This method uses ready-made instance segmentation methods to detect and crop images,thereby providing training data with clear foreground and background for the colorization model.This method uses the cropped image and the original image to train the colorization networks respectively,and obtain the final colorization result by merging the feature maps of these colorization networks.The parameters of the fusion model are also trained on the corresponding data set.The experimental results on the public data set show that this method is better than the comparison method in different quality indicators,and has achieved better colorization results.