Video Shadow Removal Method Using Matching Regions Guided By Illumination Transfer

With the rapid development of the artificial intelligence industry and the substantial increase in network speed,leading to explosive increase of images and videos.Under the light source,occlusion by inevitable obstructions will cause shadows in many videos.Removing shadows from the video makes the video more suitable for a variety of computer graphics applications,such as video segmentation,video detection and recognition,and video intrinsic decomposition.Video shadow removal is designed to remove shadows in the video and properly fill the appropriate lighting information at the appropriate location to maintain the overall continuity of the video.In order to solve spatio-temporally incoherent problem of traditional shadow removal methods for video captured by free moving cameras,a shadow detection and removal approach was proposed using matching regions guided by illumination transfer.Firstly,input video was segmented using Mean Shift method based on Scale Invariant Feature Transform,and the video shadow was detected by Support Vector Machine classifier.Secondly,the input video was decomposed into overlapped 2D patches,and a Markov Random Field for this video was set up,to find the coherent correspondences between the shadow and lit patches via patch matching guided by motion flow.Finally,in order to get spatio-temporally coherent results,each matched shadow and lit pair were conducted by local illumination transfer operation and global shadow removal.The experimental results show that the algorithm obtains higher accuracy and lower error value.The comprehensive evaluate metric is average improved by 6.32%,and the RootMean Square Error is average reduced by 30.12% compared with traditional methods based on illumination transfer.Compared with previous works,the proposed algorithm can obtain better shadow removal results more efficiently.In summary,the main contributions of this paper are three-folds:(1)Aiming at the video captured by the free moving camera,a video shadow detection method that is robust to illumination,rotation and scaling is proposed.(2)Develop a novel coherent patch matching approach to find efficient correspondences between shadow and non-shadow regions in the video.(3)Present a new shadow removal system based on illumination transfer optimization for videos captured by free moving cameras,which can produce spatio-temporally coherent and shadow-free video results.