Research On Methods For Abnormal Event Detection In Surveillance Video Via Deep Learning

With the urgent needs of social public safety,intelligent medical analysis,intelligent traffic management and smart city management,video surveillance systems have been widely deployed in airports,railway stations,roads,shopping malls,hospitals and schools.At the same time,due to the rapid development of cameras,sensors and memory,as well as the reduction of costs,the global monitoring video data volume has increased by more than a thousand PB everyday.Therefore,how to detect abnormal events in time that are inconsistent with most normal events from massive monitoring video data has become a hot topic in the field of computer vision and pattern recognition.At present,due to the problems of the traditional methods such as the definition of anomalies,cost of labeling the training samples,poor generalization and poor real-time performance,it is increasingly difficult to meet the growing demand for detection of abnormal events in videos.Since 2012,deep learning has achieved remarkable results in many fields such as computer vision,image recognition,speech recognition and natural language processing.By simulating the multi-layered structure of the human brain,deep learning tries to learn the multilayer neural network structure composed of multiple non-linear changes to realize the abstraction of data from the bottom to the top level,in order to tap the complex structure in the massive data.In this dissertation,we propose to detect video abnormal events via deep learning.Based on the extent to which the labels are available,it conducts research on three aspects of the following three modes: specific events,novelty events detection and unsupervised abnormal events detection.The research results are as follows:1.For the case of both normal samples and abnormal samples are available for the training stage,taking falling events as an example,a specific event detection method based on deep convolutional neural network(CNN)is proposed.This method divides the complete fall event into four action phases: standing,falling,fallen and not moving.The occurrence of a fall event is judged by a “standing watch” for a situation consisting of the four sequential phases.Firstly,the trimmed video clips of four phases in a fall are converted to four categories of so-called dynamic image to train a deep CNN that scores and predicts the label of each dynamic image;then,the test video is converted into a set of dynamic maps and predicted by trained CNN.Finally,the occurrence of a fall event is detected based on the observation of the four phases.The experimental results on the four falling datasets demonstrate that this method achieve real-time,high-sensitivity and high-specificity detection results.After simple modification,this method also achieved high-precision detection results on two violent detection data sets,which proved that the method has strong generalization ability.2.For the case that only normal samples are provided in advance,a novel event detection method based on Gaussian Mixture Full Convolution Variational Autoencoder Network(GMFC-VAE)is proposed.The insight that motivates this study is that the normal samples can be associated with at least one Gaussian component of a Gaussian Mixture Model(GMM),while anomalies either do not belong to any Gaussian component.The method is based on Gaussian Mixture Variational Autoencoder,which can learn feature representations of the normal samples as a Gaussian Mixture Model trained using deep learning.A Fully Convolutional Network(FCN)that does not contain a fully-connected layer is employed for the encoder-decoder structure to preserve relative spatial coordinates between the input image and the output feature map.Based on the joint probabilities of each of the Gaussian mixture components,we introduce a sample energy based method to score the anomaly of image test patches.A two-stream network framework is employed to combine the appearance and motion anomalies,using RGB frames for the former and dynamic flow images,for the latter.We test our approach on two popular benchmarks(UCSD Dataset and Avenue Dataset).The experimental results verify the superiority of our method compared to the state of the arts.3.For the case that no additional labeled training samples are provided,an unsupervised detection method for abnormal events based on Perceptual Generative Adversarial Nets(Perceptual GAN)is proposed.This method utilizes the competition between Generator and Discriminator in the Generative Adversarial Nets(GAN).The generator constantly learns how to generate abnormal samples,and the Discriminator learns how to detect anomalies.First,the initial frames are assumed that does not contain any anomalous event or only a few anomalous events,and used for training the Perceptual GAN.And then Perceptual Loss is introduced to further enhance the performance of the Discriminator and only the Discriminator is employed to detect anomaly.Finally,the detected Perceptual GAN is fine-tuned by the detected normal event to update the detection model.Experiments on three public datasets demonstrate competitive performance of this method.