Understanding the Spatio-Temporal Structure of Human Actions

The analysis of human activities has become a core interest domain in recent years, with a wide range of applications including video indexing, human-machine interfaces and surveillance. Based on the particular application the input source could be obtained from motion capture systems, depth cameras or video cameras. Regardless of the input source, the key challenge in human action recognition lies in explicitly or implicitly modeling the temporal evolution of the data.;Motivated by this challenge, in this thesis we develop techniques for human action recognition and focus on analyzing the spatio-temporal structure of the data. We examine the decomposition of human action to elementary motions and exploit the discriminative power of inference techniques that account for the temporal ordering of the data. Next, we show that capturing the temporal evolution of the human motion explicitly through dynamical systems leads to an accurate unsupervised segmentation of different actions. However, the above methods work best for constrained scenarios such as static cameras. Consequently, for the case of video sequences in uncontrolled environments, we introduce spatio-temporal features that encode the temporal evolution of local events. Based on these features, we construct a middle-level representation of the video and present a structured action classification model.