Tracking and activity classification in video surveillance applications

Automated video surveillance is a field in rapid expansion. Our research goal is to build a 3D person tracker for indoor video surveillance. In order to put our work in context we review various vision-based techniques for tracking people and detecting abnormal behaviours. The approaches are presented in bottom-up fashion starting from low-level algorithms to segment foreground objects, to trackers and action recognition systems. We also present a model-based camera localization technique. Our contribution is a 3D tracking system that can be subdivided into three modules: background subtraction, camera pose estimation and the tracker itself. The background subtraction algorithm uses the Discrete Cosine Transform coefficient blocks of JPEG encoded images as observations to compute the most likely state of each block with a Hidden Markov Model. Camera pose estimation is implemented as an edge-based CAD model registration technique using a particle filter. Finally, the tracker uses the registered model to perform 3D tracking from monocular images by assuming that the feet of people touch the floor. It's able to estimate the position and speed of tracked people at 2Hz, the maximum frame rate of the network camera. This tracker is in fact a blob-based tracker combined with a particle filter estimator. Preliminary results demonstrate that this system works well in its application context: hallway monitoring. Existing systems that use the output of trackers to recognize activities are reviewed. Finally we propose a few ideas on how to use a 3D tracker to classify scenes in order of interest with the most unusual first.