Research On Human Motion Tracking And Reconstructing From Video Sequences

Video-based motion analysis is a hot topic in the field of computer vision. The research results can be widely used in many application areas, such as "smart" surveillance system, advanced user interfaces, motion analysis in sports training, and virtual reality, etc. Monocular videos are conveniently available to general public, and all kinds of movie, physical motion, and dance are stored in monocular video format. Hence, human motion tracking and analysis from monocular video is a very attractive idea. The depth value of an object will lost when the object is projected onto a 2D image plane. Therefore, 3D motion reconstruction from 2D motion sequences is still a challenging task.According to the summary and analysis of the relevant research works in literature, we make research on video segmentation, monocular video-based human motion tracking, 3D human motion reconstruction, and animation. Our contributions are summarized as follows:1. We proposed an algorithm for image segmentation by SMC and prior probability modelA common method for real-time segmentation of moving regions in image sequences involves "background subtraction". When the pixel value of the background obeys a Gaussians distribution, it is a good method to extract moving objects using 3σrule. Althougth the background can be extracted completely by this method, but the foreground will be segmented as the background while there are some intersection of pixels value between the foreground and the background. This paper extends the original background subtraction method by using Particle Filtering and prior probability model to predict an anticipated foreground district for a coming frame, and an adaptive threshold value which is obtained according to the predicted value will be used to segement moving objects. The proposed algorithm can reduce the error of the pixels of foreground to be segmented as pixels of background.2. We proposed an algorithm to estimate the relative 3D coordinates of joints from their corresponding 2D image coordinatesIt is an ill-conditioning problem to estimate the relative 3D coordinates of joints from their corresponding 2D image coordinates without prior knowledge. Although we can estimate the relateive 3D coordinates of joints under scaled orthographic projection by using some constrains, but the reconstructed 3D coordinates of joints will deviated from the ground truth, due to that the scaled orthographic projection is only an approximation of the real camera model. In this paper, the relateive 3D coordinates of joints are determined under perspective projection, and the Euler angles of joint are estimated by inverse kinematics. Comparing to the traditional methods, the advantages of proposed method include fewer constraints, without knowing the parameters of camera model, easy to implement and more precise performance of the pose reconstruction.3. We proposed an algorithm of markerless human motion tracking for monocular video sequencesAs the depth value of an object is lost when the object is projected onto 2D image plane, the traditional algorithm of markerless human motion tracking can only track the 2D coordinates of objects rather than reconstructing the 3D pose of human. Moveover, due to the absence of obvious markers, the tradiational method can not handle the longer video sequences. To overcome these problems, we propose an algorithm of markerless 3D human motion tracking for monocular video sequences which has some advantages as follows: 1) the deformable appearance template corresponding to a given joint candidate on the 2D image plane is obtained based on forward kinematics and inverse kinematics, then the 3D human pose is reonstructed by template matching. 2) We reconstruct 3D human pose in depth-first order, and the optimal human pose is obtained by local search, the proposed method can track the longer video sequences.4. We proposed an algorithm of 3D virtual human animation based on H-animThe virtual human geometry is modeled according to H-anim standard defined in VRML. In the process of importing virtual human files, we use FSM (Finite State Machine) to parse them. Furthermore, a tree structure for modeling the geometry hierarchy relationship within virtual human is developed in the purpose of making virtual human display and control more convenient. Based on these improvements, a SDK named VHA is designed for the animation of virtual human. Additionally, we proposed an algorithm of 3D virtual human animation based on the tracked motion data. The virtual human animation created by the proposed method is not expensive and high plausible. The proposed algorithm can be widely used in many applications such as virtual-reality, computer game, and human animation.