| The purpose of grayscale image colorization is to restore a single channel or gray image into a three-channel color image and helps the image present more abundant visual information.It is of great significant in remote sensing mapping,security monitoring and film and television production.Due to the same gray value corresponding to a variety of different colors,grayscale image colorization is an ill-posed problem.In order to improve the quality of colorization results,researchers in this field introduce a reference image as the exemplar of colorization.The existing methods mainly focus on the calculation of global feature similarity and the application of global semantic information,making the incorrect colors on instances.To address this problem and make the result more refined,this paper proposes an instance-level exemplar-based colorization model.The model leverages instance information to generate pairs of instance-level reference images and grayscale images automatically,and then realize global and instance two-level color transformation.Besides,to further fuse the two-level color results,the color refinement module is introduced to fix the incorrect color and smooth the color to obtain results with more details.If the image colorization algorithm is directly applied to video colorization,it will produce flicker and inconsistent color.To keep the color consistency between frames,this paper proposes an exemplar-based video colorization model with spatial and temporal attention mechanism.Firstly,the model calculates the spatial similarity between the input gray frames and the reference images to realize the color transformation.Based on these results,the quadratic weighted fusion is performed on the temporal dimension.In addition,color edge information is introduced to constrain the range of color generation in the process of image reconstruction,so as to obtain results with suitable color and rich details.In this paper,abundant comparison experiments and ablation study are designed and implemented to verify the effectiveness of the above two proposed colorization algorithms.Quantitative evaluation and visual comparison results show that the proposed image and video colorization algorithms have better performance than the existing colorization methods. |
