Probabilistic models of actions and interactions

In this thesis, we address the problem of developing efficient probabilistic models of actions and human-object interactions. The common theme throughout this thesis is the combination of various types of visual information to improve recognition of activities in video. We present a probabilistic modeling approach for representing human motion in a concise and descriptive manner. Our approach works by combining motion data and static information within a single probabilistic integration framework. We build a higher level representation that can be described as "constellation of constellation models", and combines models of individual human poses with the model of motion dynamics.;Next, we address a novel problem of automatic vision-based recognition of atomic human-object interactions. We introduce a concept of static actor-object states as descriptive instantaneous configurations of actor and object that correspond to the moment of physical contact. We consider actor-object states at the detail level of human body-parts, and delve into deterministic approaches for discovering these actor-object states. Here, we present a pattern discovery algorithm that extracts sequences of static-states by matching data-sequences extracted along tracked body-part trajectories. We present a novel dataset of primitive actor-object interactions, and obtain validation of the concept of actor-object states for the purpose of interactions recognition.;Having set the conceptual foundations for interaction recognition, we take our analysis further by proposing a probabilistic model of human-object interactions using actor-object states at the detail level of entire body. We develop a novel Expectation-Maximization learning algorithm for discovering constrained actor-object states in unsegmented videos. This approach integrates key-frame-based representation within a part-based probabilistic framework. Additionally, we apply the proposed algorithm to the problem of learning human motions from unsegmented sequences.;Finally, we generalize our probabilistic modeling framework to the models in multiple projection spaces. We propose a general approach for integrating different types of information available for general visual phenomena. We develop a learning method that estimates simultaneously the structure and parameters of these integrated models. The model learning procedure is accomplished by maximizing the Bayesian Information Criterion within the setup of the Expectation-Maximization algorithm. We evaluate the proposed method on the problem of recognizing actions and interactions. We also cast the object recognition problem within the proposed framework, and show how multiple cues can be integrated into the unified model of target object.;The extensive experimentation reported in this thesis suggests that combination of different types of information indeed improves action recognition. Additionally, the proposed concept of actor-object states proves to be promising for recognizing human-object interactions.