Research On Image Colorization Algorithm Based On Classification Loss Function

As the main branch of image restoration,image colorization is an important research direction of computer vision and plays a great role in processing grayscale images such as medical images,night vision images,electron microscope images,satellite remote sensing images and old photos.Although the traditional image colorization algorithm has achieved good colorization effect,there are still many problems in colorization.On the one hand,the traditional colorization methods that require manual work cannot meet the colorization need of the growing and gradually complex grayscale images.On the other hand,the problems of how to effectively improve the colorization accuracy,saturation and the colorization coherence between complex image objects and background have also been troubling domestic and international scholars.In recent years,image colorization algorithms based on deep learning have achieved better colorization results.Deep learning-based colorization algorithms can achieve not only automatic colorization of grayscale images,but also the better colorization effect.However,due to the uncertainty and diversity of image colorization,the image colorization algorithm based on the regression loss function has the problem of unsaturated colorization effect.Although the current colorization algorithm based on classification loss function can solve the problem of unsaturated colorization effect,the algorithm is difficult to train due to the excessive computation and has the colorization overflow problem.Therefore,based on the study of many domestic and foreign image colorization algorithms,this paper gradually improves the image colorization algorithm using classification loss function for the problems that exist,and the main research contents and innovative results are as follows:1)To solve the problems of difficult training,inaccurate and overflow colorization in the traditional classification loss function image colorization algorithm,this paper proposes a classification loss image colorization algorithm combining classification subnetwork and asymmetric fusion module.The traditional method of calculating color categories requires complex mathematical calculation of each pixel’s color value to obtain the corresponding color categories and balance weights,resulting in huge computational effort.To solve this problem,this paper constructs the category transformation matrix to get the corresponding color category matrix by index the color value matrix of the real image and constructs the color category balance matrix to get the corresponding weight matrix by index the color categories matrix of the real image.This method can significantly reduce the computational effort and optimize the network training.When the image colorization algorithm colors grayscale images,the lack of category information of the images will seriously reduce the colorization accuracy.To solve this problem,this paper constructs a classification sub-network and introduces a category loss based on the 1000 color categories classified by the Image Net training set so that the encoder can obtain more accurate global features and thus improve the colorization accuracy.At the same time,this paper introduces the asymmetric fusion module to fuse the multi-scale features of U-Net encoder,which can capture both global and local features of grayscale images and effectively suppress colorization overflow.The quantitative experiment shows that this colorization algorithm achieves PSNR,SSIM and Colorful metrics of 25.8802,0.9368 and 23.5108 for 50,000 validation set images of Image Net dataset,respectively.The qualitative experiment shows that this colorization algorithm significantly reduces the computation,effectively improve the coloring accuracy,and significantly suppress coloring overflow.2)To address the colorization consistency problem caused by the image colorization algorithm extended to the pixel-level image colorization algorithm,this paper proposes a pixel-level classification loss image colorization algorithm incorporating the attention mechanism.Although the image colorization algorithm in the previous chapter achieves excellent colorization results,the colorization consistency problem arises when the algorithm is extended to pixel-level networks because the algorithm does not highlight the object information in the images.To solve this problem,this paper constructs the intensive residual attention fusion module to enable the image colorization network to extract the location of objects more comprehensively and accurately in the image and the surrounding structural information related to the objects,to eliminate the colorization consistency problem and improve the colorization effect.In addition,this paper constructs the asymmetric attention fusion module to enable the network to obtain more accurate fusion features and improve the ability of the image colorization algorithm to suppress colorization overflow.The quantitative experiment shows that this colorization algorithm achieves PSNR,SSIM and Colorful metrics of 26.1039,0.9402 and 24.0264 for 50,000 validation set images of Image Net dataset,respectively.The qualitative experiment shows that this colorization algorithm significantly solves the coloring consistency problem.3)To solve the problems of local incoherence in the background of color images and inaccurate colorization of small objects,this paper proposes a semantic segmentation-based pixel-level classification loss image colorization algorithm.Although the previous image colorization algorithm generates better colorization effect,the algorithm ignores the colorization of the image background and the coherence of the image,resulting in local incoherence of the generated color image background and inaccurate colorization of small objects.To solve this problem,this paper proposes the semantic segmentation sub-network to distinguish and judge each object and background in the input image based on the previous algorithm and guide the network colorization according to the semantic loss to significantly improve the accuracy and coherence of each object and background colorization.Next,this paper counts the color category distribution in the COCO dataset and reclassify the color categories and calculate the corresponding category conversion matrix and category balance matrix based on the updated color categories.The quantitative experiment shows that this colorization algorithm achieves PSNR,SSIM and Colorful metrics of 27.0254,0.9425 and 25.1364 for 5000 validation set images in COCO dataset,respectively.The qualitative experiment shows that this colorization algorithm significantly solves the problems of uncoordinated local coloring and inaccurate coloring of small objects.