Detection, Tracking, and Identification of People using Stereo Vision

This thesis explores how to use real-time stereo vision to detect, track, and identify people in a surveillance scenario. The introduction of stereo vision significantly impacts this traditional computer vision application, which is currently dominated by monocular sensors. The additional depth information provided by stereo vision brings the object detection, tracking, and identification problems from the 2-d image domain to the 3-d world domain. This translation results in dramatic performance improvements. To enable this approach, several problems need to be solved.;First, the system must perform in real-time. Surveillance systems usually operate without interruption, 24 hours a day. If the incoming video cannot be processed in real-time, the unprocessed data accumulates and eventually overflows the storage system. In addition, surveillance systems often need to respond immediately to certain events which require prompt handling. This thesis proposes a GPGPU-based stereo algorithm to deliver real-time performance on fairly high resolution.;Second, detecting objects from video data usually involves a background model which estimates the behavior of the background scene. In the case of a stereo camera, a model which combines both appearance and depth information is proposed. This new stereo-based background model not only detects foreground regions from dynamic and cluttered backgrounds, but also detects moving shadows with high confidence. Based on this enhanced background modeling, object detection and tracking are performed on a top-down plan-view map, through which the uncertainty due to occlusion is handled with ease.;A new approach to identifying person is also proposed so that tracking and searching individuals in a sparsely deployed camera network is possible. This new approach is based on a selective collection of 3-d enhanced local image features detected from a certain object, along with an efficient representation of the spatial structure relations amongst those features. Compared to existing approaches, this approach shows significantly improved performance especially on objects with flexible structure. Finally, a multi-resolution object descriptor compression scheme offers a solution to the problem of scalability in the proposed people identification algorithm.