The Study Of Foreground Object Detection In Video Surveillance

In recent years,video surveillance has played an important role in industrial production and security monitoring.Its main feature is the use of machine vision methods to acquire video(or image sequences)from sensors,and automatically calculates and analyzes the surveillance scenario without human interference.Detecting,tracking,localizing and identifying objects in the dynamic change scenarios,analyzing and judging the behavior of these targets in order to achieve both regular monitoring and timely occurrence of abnormal events.Foreground object detection belongs to the pre-processing part of machine vision,which is a key prerequisite of various subsequent advanced video processing tasks such as object tracking,object recognition,video coding,video analysis,and so on.In recent decades,the foreground object detection technology has been extensively studied by researchers all over the world.A large number of algorithms have been proposed and used in real applications,which makes considerable progress in this area.However,due to various complex challenges appearing in the real-world scenarios,such as dynamic background,illumination variations,camera jitter,shadow and camouflage,designing a real-time and robust foreground object detection algorithm is still a major challenge.This paper takes surveillance video as the research object and conducts basic algorithm related research on foreground object detection.The purpose of our work is to provide theoretical and practical reference for the real applications of video surveillance system.The main innovative research work and results are shown as follows:1)A variety of new texture feature operators have been proposed for foreground object detection.Color feature is the most widely used feature in various research fields of computer vision,but in the surveillance scenarios,which involve challenges such as illumination variations,camouflage,and shadows,color feature shows great limitations.So,many classic texture features such as LBP(Local Binary Pattern),LTP(Local Ternary Pattern),and LDP(Local Difference Pattern)are proposed and applied to foreground object detection.These texture features show great robustness for illumination variations,shadows,etc.However,the video captured in the surveillance scenarios may also contain many other challenges: noise,camera jitter,dynamic backgrounds,and so on,the texture features perform poorly when deal with these challenges.Therefore,we consider improving the existing texture features from many aspects.At the same time,compared with the use of only color features or texture features for foreground detection,we made a combination of the color features and texture features for background modeling,so that they can compensate each other for better performance.Experimental results show that compared with other algorithms,our algorithm shows better performance in precision and processing time.2)A Multiscale Fully Convolutional Network(MFCN)is proposed for foreground object detection.In recent years,the Convolutional Neural Network(CNN)has made breakthroughs in various fields of computer vision.Some researchers also have applied CNN to foreground object detection.The core idea is for the current input image,a small image patch is extracted around the pixel,feed the image patch into the trained model,and outputs the classification result(foreground/background)of the corresponding pixel.However,the patch-based methods have several drawbacks,the prediction speed is very slow,the calculation is very redundant and the accuracy is not high enough.The proposed Fully Convolutional Network(FCN)provides an end-to-end prediction method for the pixel-level classification task.We try to apply FCN to foreground object detection.The experimental results show that the FCN architecture is better than the patch-based one.The FCN has a significant improvement in speed and accuracy.Then,we borrow some new ideas of network architecture design appearing in recent years,and proposed a multiscale fully convolutional network for foreground object detection.Experimental results show that the proposed algorithm achieves state-of-the-art performance in the public dataset,while operating at real-time.3)A novel real-time foreground object detection framework is proposed by combining traditional foreground detection algorithm and semantic segmentation algorithm.Although existing CNN-based foreground detection algorithms can achieve high precision,but these algorithms require a model training process for each surveillance scenario,so they have some limitations in real-world applications.This paper proposes a novel foreground object detection framework that combines the traditional foreground detection algorithm with the real-time semantic segmentation algorithm.The framework has no need to train the model for each surveillance scenario,and it belongs to the unsupervised foreground detection algorithm.The core idea is to improve the modeling ability of foreground detection algorithms by combining semantic information.In the real-world surveillance scenarios,many challenges such as such as shadow,noise,and dynamic backgrounds often occur.These challenges greatly affect the modeling accuracy of foreground detection algorithm,which causes many wrong foreground detection results.However,semantic segmentation algorithms are very robust to deal with these challenges,so we consider using semantic segmentation results to help dealing with these challenges to obtain more accurate foreground detection results.Experimental results shown that our algorithm outperforms all unsupervised foreground detection algorithms and achieves real-time requirements.4)A convolutional neural network-based foreground detection results fusion strategy is proposed.In recent years,various foreground detection algorithms have been proposed by academia and industry,but these algorithms all have their own advantages and disadvantages.Few algorithms can simultaneously deal with various challenges appearing in the surveillance scenarios.Therefore,researchers consider combining the results of various algorithms to take the advantages of different algorithms.The majority voting-base and genetic algorithm-based fusion strategy has emerged.This paper proposes an fully convolutional encoder-decoder network architecture for foreground detection results fusion,which generates more accurate foreground detection results by inputting various foreground detection algorithm results into the network.The experimental results show that the fusion strategy proposed in this paper is obviously superior to other fusion strategies.