Fast Grayscale-thermal Foreground Detection With Collaborative Low-Rank Decomposition

Moving object detection is an important yet challenging problem in computer vision,and plays a crucial role in variety of vision applications,such as visual surveillance,self-driving systems and robotics.In recent years,moving object detection has achieved remarkable result in computer vision,and some methods have been applied in wide projects.Under some extreme conditions,such as low illumination,severe smog and other harsh environments,these moving object detection algorithms that use single modality cannot work well due to the limitations of imaging quality.The defects of the visible light modality are compensated for by introducing the thermal infrared data.Therefore,grayscale and thermal data can complement information to each other to achieve more robust moving object detection in challenging scenarios.We study the problem of multi-modal moving object detection in challenge environments,and the major works are as follows.First,we propose a novel approach,called Collaborative Low-Rank Decomposition(CLoD),to accurately detect moving object by fusing information between different modalities.Specifically,given two data matrices by accumulating sequential frames from the grayscale and the thermal videos,CLoD detects the foreground objects as sparse noises against the backgrounds with collaborative low rank structure.For general multi-modal methods,it inherently indicates that available modalities are independent and contribute equally.This may significantly limit the performance in dealing with occasional perturbation or malfunction of individual sources.In order to improve the problem,in this dissertation,We impose the low rank constraints on the joint background matrix that concatenates all matrices of different modalities,and also incorporates modality weights to achieve adaptive fusion of different source data.For the optimization,unlike most methods that usually relax the rank constraints on the background matrix to the nuclear norm,which involves one time-consuming SVD operation in each iteration,we decompose the background into two sub-matrices,and make the efficiency of the algorithm has greatly improve.Extensive experiments on the recently public benchmark GTFD suggest that our approach achieves comparable performance in terms of both accuracy and efficiency against other state-of-the-art methods.Second,in order to further improve the efficiency of CLoD,we develop a more efficient method.we first partition the input video into a set of blocks,in which each block consists of several temporally adjacent and overlapped image patches.Block-based processing method can suppress video noises effectively,and also reduce the computational burden significantly.Meanwhile,we can adopt many manners to represent each block,such as color feature,the value of pixel,gradient feature.In this dissertation,for efficiency,we adopt the mean value of pixels in each block to represent block feature in each modality.Then,we employ the edge-preserving filtering method to achieve the pixel-level accuracy with reliable modality(higher modality weight)as guidance.Extensive experiments on the recently public benchmark GTFD suggest that our approach achieves comparable performance in terms of efficiency without much loss in accuracy against other state-of-the-art methods.