Study Of Deep Palette-based Color Decomposition For Image Recoloring With Color Harmonization

Image recoloring,as a branch of computer vision and image processing,has many significant applications such as old photo restoration,video special effects,photo re-touching and graphic design.Although there are many image processing software and methods that can achieve color editing of images,they cannot balance functionality and convenience.Therefore,it remains challenging to present a framework that is suffi-ciently fast,user-friendly,and generates recolored images with complete details and harmonious color.In order to address the existing problems and downsides,this thesis presents a deep palette-based color decomposition network for image recoloring with color harmonization.Most palette-based image recoloring methods solve palette extraction and image decomposition separately,and cannot achieve an end-to-end joint optimization solution.In addition,these methods are time consuming and may cause some artifacts with unnat-ural or less vivid apperance.Different from existing methods,the thesis propose a light-weight CNN model for image recoloring in an end-to-end manner.A fully point-wise network is presented to map continous color space to a compressed discrete palette space with a pixel scrambling strategy,and multi-scale module.Consequently,a DCD-Net constraint by a region prior loss is proposed to achieve image decomposition.Extensive evaluations demonstrate the superiority of the proposed method over state-of-the-arts.Furthermore,a novel constraint derived from color harmony theory is introduced to guide the generation of aesthetically pleasing recoloring results.Firstly,a template matching algorithm is proposed,which can automatically and correctly find the best matching harmonic schemes for images.The saliency estimation is also added to the template matching process,so that the color distribution of the optimized image can try to keep the color of the saliency area unchanged,and achieve color harmony by changing the color of other areas.Then,shift the hues to a harmonic position according to a certain harmonic template.In this thesis,two methods are proposed to solve the problem of inconsistent hue conversion which is in the existed methods.The first is to add super-pixel segmentation to perform uniform hue conversion on the pixels in the super-pixel block,so as to achieve a continuous natural transition of colors in local areas.The second method is based on the the harmonization of palette color,which is extracted by PE-Net.This method can not only greatly reduce the calculation time,but also generate global and local consistent color harmony,retain the details intact,and the color transition is natural.The experimental results show that the image color harmony algorithm proposed in this thesis can provide aesthetically pleasing recoloring results very conveniently for users.