Model-based human pose estimation and tracking

Estimating human body pose in static images and tracking pose in monocular video are important for many image and video understanding applications including video surveillance. This is a difficult problem due to the presence of image noise, ambiguities in image observation, high dimensional state space and partial occlusion of the human body. The challenge is to handle realistic scenarios where the person may appear in arbitrary posture and without using special markers on the human body or clothing.; A model-based framework is proposed for human pose estimation. A human body model is built to represent important characteristics of human pose, shape and clothing and it is used for pose estimation in an analysis-by-synthesis approach. With an object-centric human model, the proposed method is relatively less sensitive to different camera viewpoints.; A statistical inference technique based on data-driven Markov chain Monte Carlo (DD-MCMC) is used to estimate the pose. Body components such as face, shoulders and limbs, are detected using various image cues including pattern, color and edges. A hierarchical approach is also developed to extract 2D positions of limbs efficiently using edge features such as parallel lines and shape contours. The results of these component detections are used to generate data-driven hypotheses of the pose parameters so that the state space can be searched more intelligently and efficiently. To track multiple people in the scene, a multi-level hierarchical state representation is used to estimate their poses in a coarse-to-fine manner.; The proposed method addresses several important issues related to pose estimation and tracking. It can handle a large variation of human pose and appearance and from different viewpoints without requiring large number of training data. Different types of image cues are used to guide pose inference and this improve the efficiency and robustness of the method. This enables automatic pose initialization at the beginning of tracking as well as re-initialization of pose following tracking failure.; Experimental results on images, indoor and outdoor video sequences show that the method is able to estimate and track poses of single and multiple persons in realistic scenarios.