Learning to extract interesting events from outdoor image sequences

This work develops a framework accompanied by experimental validation for interesting event detection in real-world outdoor image sequences.; The two most important components of this thesis are a family of algorithms for robust target tracking in real-world outdoor scenes and a statistical learning approach for learning the different classes of events in a scene. Dynamic visual events can be likened to natural language sentences given the sequential nature and the diversity in their occurrence. However, the application of natural language parsing techniques such as context-free grammars to video events is non-trivial due to the difficulty in learning of such grammars from image-based data. Unlike well-defined natural language sentences, interpretation of image data is ambiguous, more so in the case of complex environments. Until now, learning is restricted to statistical parameter estimation of models with prespecified structure.; This work addresses the problem of learning video event grammars by employing semi-supervised learning. As opposed to starting from a grammar corpus, this work introduces a novel method for incrementally augmenting the grammar from the unlabeled data through bootstrapping from a small labeled data-set and an entropy minimization-based regularization. This is more realistic in generic environments. In addition, robust classification is attained through a normalized edit distance measure applied to event parsing. Experimental results for event detection in real-world scenes for a traffic: intersection monitoring application are presented.; A similar problem with event detection from image sequences is the problem of obtaining robust target localization. Tracking has been a longstanding research problem with several solutions. Non-parametric estimators are by far the best known estimators that try to attain robust tracking through sampling at the price of immense computational over-head, thereby limiting the number of tracked targets. This work addresses the problem of tracking in unknown environments with large number of targets using realistic though flexible assumptions by combining multiple cues, multiple motion models, data association, and adaptive cue combination to attain a near real-time tracking solution. Extensive experimental validation of the tracking and the developed data association methods are presented.