Video Based 3D Human Motion Recovery

Video based human motion analysis is an important research field in computer vision and graphics communities, of which the technique of recovering 3D human motion from markerless images or videos is a active subject that has immediate applications in 3D computer animation, motion capture, natural human-computer interaction, intelligent video survaillence and so on. This paper studies some related techniques as following.A novel approach is proposed that extends the classical background subtraction method to extract silhouettes from videos in real time with dynamic viewpoint variation caused by camera movement. First, manifold learning is used to model the background under viewpoint variations. Then, for each new frame, the background image corresponding to the same viewpoint is synthesized on the fly by examining the local neighborhood on the manifold, and the silhouette is extracted via background subtraction. Experiments show that our approach can efficiently extract accurate silhouettes in complex situations.We propose a new adaptive and compact silhouette feature that can be used to express the silhouettes extracted in the previous step. We first examine a series of popular shape features in the context of 3D pose recovery, getting some valuable insights on the choices of features. Then, an adaptive and compact silhouette feature is proposed by progressive feature combination and selection from traditional shape features. Compared to traditional features, the new feature is more effective for 3D pose recovery and the dimension is reduced.We propose a new generative 3D pose recovery method. First, extracted silhouettes are analyzed to derive 2D positions of spine and end sites. Then, 3D poses are recovered by optimizing an object function that encodes the correspondence between the analyzed silhouettes and a pose-parameterized 3D human skeleton. In order to reduce the computational complexity, an effective and computationally efficient object function is devised. A novel iterative optimization process that exploits the human skeleton structure is also proposed to boost the optimization. Experiments show that complex motions of a large variety of types can be recovered by the proposed method.We propose a new perceptual pose distance: Relational Geometric Distance. Distance metric of 3D poses can be directly used to estimate the performance of a 3D motion recovery system. First, an extensive relational geometric feature pool that contains a large number of potential features is defined, and the features effective for pose similarity estimation are selected by Adaboost. Finally, the selected features form a pose distance function that can be used for novel poses. Experiments show that our method outperforms others in emulating human perception in pose similarity.We propose a new example based 3D motion recovey method, and develops a real system implementation. First, a lookup database is constructed from silhouettes and corresponding 3D poses. Then, for silhouettes extracted from each frame in the video, the database is searched in a k nearest neighbor way and a list of pose candidates is returned. Finally, dynamic programming is employed to find the optimal pose path in the candidate lists. The proposed method recovers 3D motions automatically in real time.