| Automated surveillance addresses real-time observation of people, vehicles and other moving objects within an environment, leading to a description of their activities and interactions. The technical issues include moving object detection and tracking, motion analysis, camera localization and calibration, activity understanding and anomaly detection, etc. This dissertation presents the human detection and tracking, network topology inference and human activity analysis in the multi-modal sensor network.;First, the relationship between visual human motion and acoustic signals (i.e., step sound) has been explored in the framework of Bayesian network to detect and track walking humans. A hierarchical Bayesian network is proposed for detecting and tracking multiple moving objects simultaneously. The performance of the hierarchical Bayesian network has been evaluated in the real-life experimental scenarios of various complexities and compared with single modality-based methods (either audio or video only) and sensor fusion-based approaches.;Secondly, this dissertation proposes to integrates face recognition in camera network topology inference that provides robustness to appearance changes and better models the time-varying traffic patterns in the network. A continuous learning scheme based on the Monte Carlo Markov chain Expectation Maximization algorithm is also proposed to continuously learn traffic patterns in the dynamically changing environment.;Finally, the dissertation proposes a novel human activity classification approach in the camera network with non-overlapping field-of-views, which uses the human identity similarity and travel times between departure and arrival nodes to classify observed human activities into: "normal," "break-in," "stay," and "sudden appearance/disappearance." A semi-supervised EM algorithm is also proposed to alleviate the problem of limited labeled training data. |
