| Image colorization is an effective approach to provide plausible colors for grayscale images,which can achieve better and pleasing visual qualities.It has important research significance and application value in the era of increasingly large number of images.Although exemplar based colorization approaches provide promising results,they are relied on semantic colors or global colors only from the reference images.Therefore,it is only suitable for specific coloring patterns and has poor generalization performance.In this thesis we study the inefficiency of state of the art colorization methods in accuracy and generalization.The main work includes:First of all,in order to color images reasonably no matter whether the reference image is semantically related or not,we proposed a deep neural network that fuses the semantic colors and global colors of reference image for colorization,which effectively transfers the colors from reference image to the input grayscale image.Secondly,to reduce the overhead of memory and time caused by the extra semantic feature extractor,we use multi-task and attention operation on the intermediate layers of the encoder in the main colorization sub-network.Therefore,the semantic correspondence calculation and image colorization can be learned jointly to boost system’s performance.Meanwhile,in order to reduce the ambiguity between semantic colors and global colors,we proposed a gating mechanism to automatically determine the importance of them.What’s more,a differentiable histogram generation algorithm is proposed,where the color histogram loss can be applied on the ab channel generated by the proposed system directly for effective colorization.Finally,to extend the image colorization method to the video colorization task,the control of the previous image is introduced based on the nearest neighbor color transfer mechanism,and a hierarchical color fusion technology is designed.The proposed method ensures the time consistency between colored frames while maintaining the vividness and rich color of colored results.The evaluation results on public image and video datasets show the effectiveness of the proposed exemplar based image and video colorization framework and related algorithms. |
