Video Foreground-background Separation Via Spatiotemporally Assisted Clues

Video foreground-background separation is an important video analysis technique and has a wide range of application.However,due to several difficult scenarios such as bad weather,camera jitter,illumination changes and moving background,most existing methods suffer from the deficiency of accuracy.Therefore,this thesis focuses on this topic based on traditional algorithms and deep learning methods,respectively.The main research contents and contributions of this paper are depicted as below:1.This thesis proposes a unified framework called spatiotemporally scalable matrix recovery(SSMR),which has a moderate computational and space complexity scalable.In the proposed model,the inherent batch-mode nuclear norm is replaced with an explicitly low-rank matrix factorization in order to achieve online implementation.Motion information extracted by an optical flow method is incorporated into the data term to rectify data consistency.Affine transformation is embedded into the model and simultaneously optimized with other variables to handle camera motions.In addition,this thesis proposed a pyramidal scheme to achieve spatial scalability for high definition videos.At last,all variables are optimized under the framework of the alternative direction method(AMD).Experimental results demonstrate that our method outperforms many state-of-the-art methods and can handle videos of various complex scenarios.2.This thesis proposes an end-to-end cascade deep convolutional neural network.In this method,the RGB images and the associated optical flow maps are convoluted simultaneously and their feature maps are fused after each convolution layer.The followed transposed convolution blocks learn a projection from the feature space to the image space to generate a binary foreground mask.Subsequently,the background reconstruction network takes the generated mask and the current frame to reconstruct the background pixels occluded by foreground.The experimental results show that the proposed spatiotemporal fused cascade convolutional neural network has achieved better results on the public dataset than other methods.The foreground detection and background reconstruction results greatly outperform the existing state-of-the-art methods.