Study On Tracking And Capture Of Human Motion In Monocular Video Sequences

Video human motion analysis is an important research area which combines computer vision and graphics techniques. The focus of the research is in detecting and tracking figures in monocular or multicular videos, capturing and reconstructing the human motion, and then describing and interpreting the human behavior. This research has various applications in human animation, computer game, virtual reality and augmented reality, human-computer interaction, video surveillance, sports video analysis, computer-aided clinical diagnosis, and so on.This thesis focuses on the problems of tracking human and capturing three dimensional human motion in videos. This work investigates on tracking the articulated motion and evaluating the relative three dimensional motion of the figure in the form of articulated skeleton chains in monocular videos. Approaches of tracking and capturing human motion in monocular videos are proposed. This research is anticipated to help obtain the abundant motion data from plenty of videos including movie, TV and sports videos, and enable the myriad activities of human created in virtual world.On the basis of sufficient review to the related works, the thesis introduces a system designed and implemented on Video Human Motion Tracking and Capture, called VHMTC. The work in the thesis makes three main contributions in the field:· An articulated skeleton human model suitable for tracking human motion in 2D videos and 3D reconstruction of the human motion, is proposed. The 2D articulated skeleton human model with the appearance model ensures an accurate model-image matching during the tracking. The tree structured 3D human skeleton model having the same articulated structure with the 2D model facilitates the evaluation of the 3D coordinates of joints from 2D coordinates.· A novel approach on markerless human motion tracking in monocular videos isproposed. The new approach takes advantage of a learnt motion model to predict the human motion, and uses an appearance model to calculate the likelihood. The conditional density propagation technique of the particle filtering is employed to implement the human tracking in monocular videos. The capability of the particle filtering in processing non-Gaussian and nonlinear situations makes the new approach robust to the ambiguity and clutter background. With multiple hypothesis maintained, the new approach is capable of automatically recovering from tracking failures and processes the occlusion and auto-occlusion problem correctly. It uses the motion data obtained from the training video sequences, and no 3D motion capture data is required. The image processing techniques such as image difference and silhouette extraction are not necessarily required, and any markers and special background are not required either.· A new method on 3D reconstruction of the human motion from monocular videos is proposed. In many imaging situations where the depth of the figure is small with respect to the distance between the figure and the camera, a scaled orthographic projection model is employed to evaluate the relative 3D motion of the figure. The relative lengths of the segments are obtained from the appropriate frames, and the articulated human skeleton model is customized with them. The 2D joints in every frame are obtained manually. Then, the process of the new approach is implemented through parameter evaluation and optimization in each frame, iterative optimization in a series of subsequence, and 3D reconstruction of the human motion sequence. The new method does not require special equipments. No assumptions of the appearance of either the human or the background are introduced. Camera calibration is not necessary. The length of the video is not limited. It is suitable to the videos such as movies and sports videos. The new method is effective and robust. The resulting motion data can be employed to generate 3D human animation.With the system in this thesis at hand, we can select suitable videos to track and capture the human motion data and use them to imitate the huma...