3d Markerless Human Pose Estimation Based On Monocular Video Sequences

Vision based human pose estimation can be applied to a number of fields including 3D animation, video games, human computer interaction (HCI), in-telligent surveillance, computer aided sports training system, etc. Even though current marker based multi-camera human pose tracking system has been widely used in commercial field, it is too expensive and the markers at-tached to the human body more or less affect the motion of the subjects. Hence it is unsuitable for some applications such as intelligent surveillance, HCI, sports training and analysis. The research of monocular video based markerless human pose tracking is boomed because of its lower cost and wid-er application. Aiming at monocular video based markerless 3D human pose tracking and estimation, the author does researches in three major aspects.Firstly, when tracking multiple objects, prior models such as path con-sistency assumption model are established in order to handle occlusion prob-lems. However, the tracker is doomed to fail when the real trajectories of the objects are greatly distinguished from the assumption. To solve this problem, the author puts forward a definition of interactive particle filter which adap-tively selects appearance template for a particle to measure its likelihood by judging the occlusion relationship between correlated objects. The algorithm successfully solved the occlusion problem among objects being tracked. Moreover, the human body limbs can be regarded as articulated objects. By applying the proposed interactive particle filtering, the 2D motions of the limbs are able to be tracked robustly even if serious self-occlusion happens.Secondly, the author proposes a set of human body models including a constrained graph model, a 2D cardboard human body model used for grabing human body limbs'observation, and a 3D stick human body model which are suitable for rendering 3D human poses. The graph model is proposed for ap-plying particle filtering with partitioned sampling. In the graph model, each node corresponds to a joint, while each edge corresponds to an observation of a limb or a limb length constraint, so that the constraints can be easily com-bined to the probability frame work of human pose tracking. Moreover, the graph model has the same joint architecture with the 2D cardboard model and 3D stick model, hence the computation of the 2D and 3D coordinates se-quence of the joints becomes more convenient.Finally, even though particle filtering provides an effective non-Gaussian, non-linear and multi-mode measure to estimate the posterior, it requires a tremendous number of particles, which makes it too time consuming to be uti-lized for human pose tracking. Partitioned particle filtering algorithm is in-troduced and applied as the framework of tracking 2D human boday limbs. By combining with multiple features including color, edge and motion, a weighing function is constructed to measure the particles'weights. In the framework of particle filter with partitioned sampling, the author puts for-ward a new adaptive way for fusing color, edge and motion cues together to construct the weighting function of particles. In addition, a simplified Belief Propagation (BP) is developed to propagate the weights of limb observations to the corresponding particles along the edges of the proposed graph model, which is capable to make a set of particles be able to carry multiple con-straints. By applying the proposed interactive particle filtering to deal with occlusion among body limbs, a robust 2D human pose tracking method is de-veloped. Finally, the relative 3D coordinates of each joint are computed based on the estimated 2D image coordinates of the joint. Subsequently, the 3D hu-man pose is reconstructed according to the relative 3D coordinates of each joint.The study supplies the multiple objects tracking with a method for deal-ing with occlusion among objects, and applies it to 2D articulated human pose tracking for self-occlusion problem. Meanwhile, by adopting partitioned sam-pling method and applying limb length constraints on a graph model, it suc-cessfully tracks 2D human poses, and then estimates 3D human poses by us-ing scaled orthographic projection model. Experiments show the accuracy and robustness of the proposed method.