Unsupervised Diverse Colorization Via Generative Adversarial Networks

Colorization of grayscale images is a hot topic in computer vision with a long history.Most previous research mainly focuses on producing a color image to recover the original one.Which means those researches are all in supervised learning fashion and the models they proposed are all deterministic models.However,since many colors share the same gray value,an input grayscale image could be diversely colorized while maintaining its reality as well as grayscale value.So the task we solve in this paper is unsupervised diverse colorization of grayscale image which has not been explored so far.The recently proposed Generative Adversarial Network(GAN),which is trained by adversarial methods,achieves a lot in many tasks.Generative adversarial network maps random gaussian noise to target image space,so that can provide diverse images while maintain their reality.It is quite rational to apply GANs to diverse colorization problem.In this paper,we design a novel solution for unsupervised diverse colorization.Specifically,we leverage conditional generative adversarial networks to model the distribution of real-world item colors.We propose several modifications to enhance our model,including(1)using fully convolutional generator with stride 1 to replace the original auto-encoder structure,reducing the amount of calculation and parameters,(2)well-designed multi-layer condition concatenation instead of original single-layer condition concatenation to get better grayscale consistency,(3)using multi-layer noise concatenation in place of single-layer noise to enhance diversity,(4)choosing YUV color space as the color image representation,(5)choosing the original GAN loss.We carefully choose the questionaire survey way to evaluate our results.The analyzation of valid feedback from 130 humans indicates that 35% of our generated diverse results successfully fooled the participants,which means we passed the Turing test and the results we generated are highly convincible.This work not only solves the problem of unsupervised diverse colorization of grayscale images,but can also be implemented to image style transfer and even artistic designing.And this is also valuable experience in the developing of GANs.