Shadow Detection And Removal Based On Deep Learning Networks

Shadow is ubiquitous in nature.A Shadow may appear when an object blocks the propagation paths of light generated from a light source.The appearance of shadows has both advantages and disadvantages.The disadvantage is that it interferes with many existing image and video processing and analysis tasks.The advantage is that the shadow actually implies the information of the light source and objects in the scene,which is conducive to the understanding of the target in the scene.Therefore,the study of shadow detection and removal methods has both theoretical and practical significance,but it has always been regarded as a key task in machine vision.Shadow detection are divided into two categories: traditional detection based on artificially constructed features and detection using deep learning networks.However,shadow detection of traditional algorithms requires a lot of difficult optical and physics knowledge to build,which greatly hinders researchers from further research in this direction and the effect is not necessarily ideal when limited by the model.While deep learning networks can achieve end-to-end shadow detection,and only need to design a suitable extraction feature method according to the features of the shadow(such as convolution and pooling)can get excellent detection results,so the use of deep learning networks for shadow detection has gradually become the mainstream.Although the performance of shadow detection methods has been greatly improved in recent years,the existing algorithms still have the following three problems: 1)when a shadow covers an area containing both bright texture and dark texture,the detector may easily overlook the shadow part on the bright texture;2)False negatives may occur when the shadow is produced by a weak light source;3)A detector is inclined to classify the dark pixels in the image as shadow.These three problems greatly reduce the accuracy of existing shadow detection methods.In response to the above problems,we proposed Double-stream Atrous Network(DSAN)for shadow detection.In the network,a Atrous Convolution Module,a Multi-layer Atrous Pooling Module and Cross-Stream Residual Modules are designed to increase the convolution field and extract and integrate the local and global features of the image better.In the experiment,the shadow detection results of the network have improved for the above three problems.Through qualitative and quantitative experiments on two large shadow detection datasets,we prove that DSAN can obtain excellent detection results compared with other nine mainstream shadow detection methods,among which the Balanced Error Rate(BER)in the SBU dataset dropped to 6.6%,and the Accuracy reached 96.2%,both ranking first.On the ISTD data set,the BER dropped to 4.0%,ranking second,and the accuracy rate reached 97%,ranking first.We also use ablation analysis to further prove that the proposed three sub-modules contribute to the extraction of shadow features and are reasonable.Based on the Double-stream Atrous Network,we applied it to intelligent surveillance video sequences and satellite remote sensing images respectively.Since the shadows in the intelligent surveillance video sequence have the same motion properties as the detected moving targets and are sometimes difficult to distinguish due to similar colors,it will cause the false detection of the foreground target detection algorithm.Using DSAN to mark the shadows can avoid interference and greatly improve the accuracy of the target detection algorithm.Similarly,our network can also be applied to the processing of satellite remote sensing image sequences that have massive information.We selected suitable images from the existing remote sensing datasets to build a remote sensing shadow dataset and use it for training and testing.After qualitatively and quantitatively comparing the existing mainstream deep learning network and remote sensing shadow detection methods,it is found that our network has achieved a leading detection level,which proves that DSAN has strong task and scene generalization capabilities.We found that the task of shadow removal can be regarded as the task of generating similar textures to the surrounding unshaded parts for the texture of the shadow coverage in the image.The pix2 pixHD can not only generate global features well,but also modify the details.Therefore,it is very suitable for the shadow removal task.Finally,we applied the pix2 pixHD to the shadow removal task for the first time,and generated high-resolution,realistic shadow-free images from color images with shadows.After training on the two shadow removal datasets SRD and ISTD,and qualitative comparison with other shadow removal algorithms,it is proved that the shadowless images generated by pix2 pixHD are the most realistic and the details are very clear.This provides a new idea for the study of further shadow removal methods.