Algorithm Research Of Shadow Removal Based On The Cycle-GAN

Shadow is a dark area due to the straight-line light blocked by some objects,which is a common phenomenon in our natural environment.However,shadows in the scenes which are introduced in the process of collecting image data will influence the accuracy and stability of computer vision algorithms.Consequently,shadow removal becomes a critical task in computer vision.In the state-of-arts of shadow removal algorithms based on Deep Learning,the models trained with unpaired datasets has become the mainstream,and there is no doubt that the most classic model among them is MaskShadow GAN whose frame is based on the Cycle-GAN.However,the performance of Mask-Shadow GAN is restricted by the structure of its generator,which makes the Mask-Shadow GAN don’t have enough ability to generate high-quality shadow or nonshadow images in the process of image transformation.In addition,all of the existed shadow removal models or algorithms combine the shadow removal and artifact repairing as one task,which means the model should remove the shadow in shadow area while recovering the consistency of color and texture between shadow and non-shadow area.In other word,there is no model or algorithm treats the artifact as an independent task to study its solution.In the view of above problems,we carry out the following research contents:(1).In this paper,we propose a shadow removal model named U-Cycle-GAN based on Cycle-GAN,which introduces the U-Net as its generator and mask-guided strategy in Mask-Shadow GAN.U-Cycle-GAN uses the skip-connection method of UNet to improve the quality of transformed shadow or non-shadow images by feature fusion,at the same time,U-Cycle-GAN successfully establishes the one-to-one mapping relationship between shadow and non-shadow images through mask-guided strategy and adversarial loss and cycle-consistency loss in Cycle-GAN.In subjective testing results,U-Cycle-GAN leaves obvious artifacts in shadow area after removing the shadows,but the color and texture in non-shadow area are effectively protected.Meanwhile,the objective testing results of U-Cycle-GAN are better than the models of the same type which are trained with unpaired datasets.(2).For the obvious artifacts in subjective testing results of U-Cycle-GAN,we treat it as an independent task and imitate the Style-Based-Generator which has the ability of characteristic decoupling to design an artifact repairing model —Characteristic Decoupling GAN(CD-GAN)which is a CGAN actually.We achieve the target of training the shadow removal and artifact repairing simultaneously through combining the U-Cycle-GAN and CD-GAN together,and we name the new model as U-Cycle-GAN-E.Relies on the ability of characteristic decoupling,U-Cycle-GAN-E not only achieves the target of repairing the artifacts in U-Cycle-GAN’s subjective testing results without changing other features,but in objective testing results,U-CycleGAN-E also makes a great improvement compared with U-Cycle-GAN.In the comparative experiments with models trained with paired datasets and algorithms based on physical model,U-Cycle-GAN-E also shows its superiority.