Peer-to-peer tracking for distributed smart cameras

Visual surveillance systems are used in commercial, military and health care applications, public transportation scenarios, and entertainment area. Putting the burden of watching long hours of videos and detecting unusual events on the shoulders of a human operator is not an effective way of analyzing videos, since attention of an individual decreases in time. Automatic object tracking and event detection algorithms can provide accurate and reliable reports on activity from video. Moreover, as cameras become less expensive, many systems will use large numbers of cameras for better coverage and accuracy. Multi-camera systems must be scalable to increasing number of cameras. This dissertation focuses on object tracking and communication protocol design for distributed multi-camera systems, where a different processing unit is used to process each camera to provide scalability. The processing units communicate with each other in a peer-to-peer fashion eliminating the need for a central server, and thus removing a single point of failure.; We first introduce methods for recovering field of view lines and solving the consistent labeling problem. We then describe a computationally efficient and robust multi-object tracking algorithm. This algorithm deals with the merging and splitting of objects on a single camera view without sending requests to other nodes in the system, thus provides sparse message traffic. Peer-to-peer systems require sophisticated communication protocols, which should be carefully designed and verified to avoid potential problems such as deadlocks. We introduce a novel communication protocol that can handle the problems caused by communication delays and different processor loads and speeds, and incorporates variable synchronization capabilities. This protocol is exhaustively verified with no errors by using the SPIN verification tool, and the verification results are presented. We then present the Scalable Clustered Camera System, which is a peer-to-peer multi-camera system for multiple object tracking. Instead of transferring control of tracking jobs from one camera to another, each camera performs its own tracking, keeping its own tracks for each target object, which provides fault tolerance. We demonstrate the system on different scenarios captured by multiple cameras placed in different setups.; This dissertation also introduces a spatio-temporal event detection system, which lets users specify multiple composite events of high-complexity, and then detects their occurrences automatically. This system does not require any familiarity with a programming language to specify events of interests to the system. Events can be defined on a single view or across multiple camera views.; In addition, a method is proposed for temporally calibrating video sequences from unsynchronized cameras by image processing operations. Two robust search algorithms, namely the temporal calibration algorithm and the RANSAC - based temporal calibration algorithm, are presented, and their performances and processing times are summarized and compared. These algorithms can handle very large frame offsets between the sequences.