Real-Time Automated Annotation of Surveillance Scenes

Video surveillance has become of a major research topic recently due to the increasing number of potential applications in public spaces. In particular, there is a demand for automated surveillance applications that detect different types of activities related to public safety, such as in metro stations. Automated video surveillance is intended to be used as an aid to human operators by bringing to their attention certain designated events of interest.;This thesis presents a real-time video surveillance system that detects a range of activities in a scene viewed by a single color-video camera. Our contribution in this work is mainly exploiting the properties of the CIELab color space to improve the performance at the low level processing, proposing a multi-level blob matching algorithm to solve the object tracking problem, and using a hierarchy of semantics for detecting events that are of interest to public spaces surveillance.;A complete framework of a surveillance system is presented. Objects in an observed scene are modeled by blobs that are detected by means of the adaptive background modeling codebook implementation based on the work of Kim et al. [2]. The implementation uses a dynamically updated codebook in which blobs in the video are characterized in color space, while also dealing with shadows. Collections of blobs, which represent potential objects of interest, are tracked and classified in real-time. For tracking, we employ a simple correlation process based on an elaborate blob matching algorithm. The essence of this algorithm is to find the best blob collection based on matching all potential color histograms from previous frames to those obtained in the current frame. Rules are used to resolve complex cases such as ghosts, occlusion, and lost tracks. Objects are then classified as either being animate persons or inanimate objects. This is essential for providing an accurate description of the scene and drawing the correct inferences about object interaction and events. Given this description of the video, a hierarchical semantic approach is used for event detection.;The proposed framework investigates a generalized approach to detecting a spectrum of behaviours based on object interactions and trajectories. These behaviours range from simple single agent events such as loitering, to more complex interactive ones, such as people walking together. Experimental results are presented for standard available videos in order to verify the performance of the system and are compared to existing results in the literature. These results show a significant improvement both in terms of quality and speed, making a step towards a more reliable fully automated surveillance system.