| In recent years,crowd safety accidents have occurred frequently in public areas such as sports venues,festival celebrations and large business districts all over the world.To this end,each country actively promotes the construction of "safe city" projects to guarantee social public safety and reduce the loss of people’s lives and property.The foundation of the construction of "safe city" is information perception.At this stage,the most effective way to timely obtain the information of public areas is building a comprehensive security video surveillance system.Therefore,a large number of monitoring devices are widely installed in airports,railway stations,shopping malls and other public places.However,traditional video surveillance system mainly relies on manual observation,which is inefficient.Nowadays,with the development of artificial intelligence technology,research on intelligent video surveillance system has gradually become a hot topic,which can automatically analyze the content of video data and extract key information,promoting the transformation from "safe city" to "smart city".Abnormal event detection is one of the key research contents in intelligent video surveillance system,which aims to use machine learning and computer vision technology to automatically discove the events caused by monitoring targets(human or vehicle,etc.)that are significantly deviated from the regular event model in video sequences.In this dissertation,we aim at three situations of video abnormal event detection:sufficient training samples,insufficient training samples and no training samples,and conduct research on three issues:fully-supervised abnormal event detection,semi-supervised abnormal event detection and unsupervised abnormal event detection.Besides,the issue of detection of abnormal event caused by human activities is discussed in depth.The main research contents and innovations are summarized as follows:1.For the case that both normal event samples and abnormal event samples are easy to collect,this dissertation simultaneously uses normal and abnormal event samples as training data and proposes a fully-supervised abnormal event detection method based on margin metric learning.Unlike existing fully-supervised schemes that can only detect the known types of abnormal events,the proposed method can utilize the prior information of some known types of abnormal events in the training data to detect multiple unknown types of abnormal events.The main steps are as follows:Firstly,the video clips are input to the attention embedded spatial convolutional(Att-sConv)network to extract their spatial feature sequences;Then,the spatial feature sequences are input to the temporal convolutional siamese(tConv-Sia)network to further obtain the temporal features.And the quadruplet contrastive loss(QC-Loss)is designed based on the spatial-temporal features of normal and abnormal video clips to conduct the margin learning,narrowing the intra-class distance of this two types of spatial-temporal feature distributions and expanding the inter-class distance;Finally,a metric distance based anomaly detection strategy is proposed to identify anomalous events in the testing video.Experiments on Avenue and ShanghaiTech datasets show that this method can effectively utilize the prior knowledge of few known types of abnormal events to detect unknown types of abnormal events.When compared with other advances methods,the detection accuracy is improved.2.For the case that normal event samples are easy to collect while abnormal event samples are hard to collect,this dissertation only uses normal event samples as training data and proposes a semi-supervised abnormal event detection method based on spatial-temporal cascade autoencoder(ST-CaAE).Compared with the existing semi-supervised strategy that trains a single neural network as a normal event model and conducts single-stage abnormal event detection,this method designs a cascade structure ST-CaAE that includes two different networks to perform more efficient modeling of normal events,and uses the two obtained normal event models to conduct coarse-to-fine two-stage abnormal event detection.The main process are as follows:First,in the first stage,the spatial-temporal adversarial autoencoder(STAAE)network in ST-CaAE learns the hidden representation of normal event samples as a multivariate Gaussian distribution during the normal event modeling process.Considering that the abnormal event samples cannot be associated with the Gaussian distribution,an anomaly detection method based on the conditional probability of the Gaussian distribution is used for rough detection of abnormal events;Then,in the second stage,the spatial-temporal convolutional autoencoder(ST-CAE)network in ST-CaAE employs the reconstruction strategy to build a normal event model,and utilizes the reconstruction error to conduct fine anomaly detection,detecting the abnormal events that are not found in the previous stage and further improving the detection performance.The method extracts optical flow and gradient feature cuboids as video event feature representations to detect motion and appearance abnormal events.Experimental results on three public datasets validate that the proposed method outperforms traditional single-stage semi-supervised anomaly detection methods.3.For the case that abnormal events are caused by human activities,this dissertation uses human skeleton feature and proposes a semi-supervised human-related abnormal event detection method based on the memory-augmented WGAN-GP(MemWGAN-GP)model:Firstly,the pose estimation and tracking algorithm is employed to extract the skeleton trajectory feature of each pedestrian in the video sequence for describing their behavior,which is decomposed into two feature components:global body motion and local pose change;Secondly,the MemWGAN-GP network is established to jointly model the global and local components of normal skeleton trajectory as two sub-processes,exploring normal human behavior patterns;Finally,the testing skeleton trajectory is reconstructed and predicted simultaneously with the normal human behavior model,and the reconstruction error and prediction error are fused for identifying abnormal trajectories,achieving the detection of human-related abnormal events in the testing video sequence.An adversarial loss based on Wasserstein distance is designed for the training process of model to optimize its reconstruction and prediction performance.Experimental results on the Avenue and ShanghaiTech datasets and their subsets that only contain human-related anomalies demonstrate the effectiveness of the proposed method for human-related anomalous event detection.Compared with the semi-supervised algorithms based on other kinds of features,the anomaly detection speed is faster.4.For the case that both normal event samples and abnormal event samples are hard to collect,this dissertation proposes an unsupervised abnormal event detection method based on self-training spatial graph convolutional network.Based on the existing unsupervised methods that rely on human assumptions for anomaly detection,the proposed method uses regression strategy to optimize the detection results based on assumptions,and employ the spatial graph convolutional network(SGCN)for achieving the unsupervised detection of abnormal events related to human activites.Therefore,the skeleton features of all pedestrians in each frame of the unlabeled video sequence are first extracted,and the spatial graph is constructed as the representation of skeleton feature to preserve the spatial connections among skeleton joints;Then,considering that there are no manually labeled training samples,a non-parametric unsupervised algorithm iForest is employed to perform initial anomaly detection,and all extracted skeleton graphs are classified into normal and abnormal skeletons sets;Finally,the self-training regression strategy is used to iteratively update an anomaly scoring module established on SGCN with the initial sets as training data.After the iterative process,the trained module is employed to recalculate the anomaly score of each human skeleton in the unlabeled video sequence,which is used for judging abnormal skeleton individuals and update the initial detection results based on hypothesis,accordingly achieving the fine detection of humanrelated abnormal events.Experimental results on Avenue and ShanghaiTech datasets and their subsets show that the proposed method is not only superior to the existing unsupervised methods,but even better than some state-of-the-art supervised methods. |
