| This thesis is about human motion: how human motion can be measured from video input, how models of human motion can be learned from example data, and how such models can be used for analysis and synthesis tasks. There are many potential applications. Examples are new paradigms in human computer interaction, visual surveillance, video database annotation, graphics, and visual effects. In almost all interesting scenarios the subjects are in motion. During talking, complex configurations and subtle lip motions are generated. During gesturing, walking, and other actions, coarse articulated limb and body movements are generated. Depending on the kind of motion and application, different abstractions and resolutions are required. Some motions are very constrained, like speaking lips or walking styles; these can be learned from data. Other actions only satisfy very general constraints. Such constraints can be coded a priori. Recognition tasks require extracting and modeling only a few discriminative features. Animation tasks require capturing every subtle details.; Human motion can be broken into two core domains: (a) Rigid-articulated motion: Full body movements where limbs are approximated by rigid segments connected at body joints. (b) Non-rigid motion: Facial and lip motions. For both domains, rigid-articulated and nonrigid motion, we show how to measure motion in solving constrained subspace problems. For rigid-articulated measurements, the constraints are analytically derived, using twist and product of exponential map representations. For non-rigid measurements the constraints are learned from example data using linear subspace and nonlinear mixture models. Given such motion measurements, we can classify motion categories with statistical models. For example, gait categories can be learned and recognized using mixture of dynamic systems and Hidden Markov Models. Lip motion categories can be learned and recognized using Multi Layered Perceptrons and Hidden Markov Models. We also show how these techniques can be applied to animation tasks. |
