Human Motion Posture Reconstruction And Tracking In Multi-camera Environment

The capture of human body posture and its posture analysis is one of the hottest research topics in the field of computer vision. It has great potential in various applications such as movies&games, surveillance analysis, human machine interaction, virtual reality, medical diagnosis and motion analysis. However, there are still many difficulties such as the non-rigid body shape, the ambiguity of2D to3D projection, self-occlusion, recovery of high-dimensional parameters and the feature extraction and matching under real situations. There are still many theoretic issues about the recovery of human body posture from digital video. Therefore, the research on human body posture analysis is of both theoretic and practical value.Reconstruction and analysis of human body posture is one of the bases of human body posture analysis. This thesis focuses on human body posture analysis, including the reconstruction and tracking of human body and hand posture. It researches on the human body location and posture tracking under multi-camera environment, which provides theoretic basis and systematic framework for human body posture capturing and analysis. In detail, the main work and contributions are summarized as follows.First, a novel algorithm which combines depth and skin information and can extract fore-ground target object in the complex scene is proposed. To segment the fore-ground image in the complex scene, the depth segmentation method is an effective method to avoid the interference in the complex scene and moving objects in the background. With the RGB camera joint with Depth camera, a segmentation method combined with depth information and skin color information is proposed. The depth information help to get clear fore-ground target object. The method is robust and efficient even in the background there have the complex scene and moving object.Second, according to the characters of pose structure, and combining the hierarchy strategy, an auto-initialized algorithm which used in indirect model human model is proposed. The general methods retrieving the pose parameters are matching a fixed pose prepared beforehand which has known the parameter, or using the learning method, or shape controlling parameters, or hierarchical strategy. We combine hierarchical strategy to propose Vector Synthesis Analysis (VSA) method and improve the voxel thinning method. The auto-initialized algorithm builds the human skeleton and topology which is matching the real man. We construct adaptive model method instead of general method.Third, when a rotation angle is decomposition three sub-angles around XYZ axes respectively, the traditional methods is to make a hypothesis that one of the sub-angles must be zero. A construction of decomposition method which is used in solving the problem is proposed in the paper. If a rotation angle that a point rotates around another point need to be decomposed into three sub-angles around axes XYZ respectively, the general method is to reduce the number of dimensions that make a hypothesis that one of the sub-angles is zero, and then solve the other two rotation sub-angles. The process is not exactly describing the motion of the joints. In order to solve the problem, we introduce decomposition method. The rotation matrix R is orthogonal matrix. There are existing matrixes U and Q to make R=QUT Selecting the appropriate matrix elements, the decomposition method can decompose arbitrary rotation angle into combination of three sub-angles rotating around XYZ axes. The method effectively represents the rotation information of each joint.Fourth, according to traditional motion model that exist the problem of high dimension of Degree of Freedom (DOF), a new motion model which has the lower dimension of DOF is proposed, and we use it to implement the posture parameters recovery. Rationally using the physiological motion principles and the characters of markless motion, we redesign the human motion model and decrease the quantity of DOF to lower, which make help accelerating to recover the pose parameters. And we also use a motion capsule concept. The motion capsule inversely drives the joints of skeleton which have to change with translation and rotation angle. We use multi-cue tracking technology which combined with silhouette cue and overlap cue of motion capsule. We used local gradient-based method to recover the optimal parameters and update the pose with them.Fifth, a framework of markless human posture capture system is constructed based on above-mentioned work. We designed a1D calibration rod to calibrate the intrinsic and extrinsic parameters which make calibration easier and quicker. And we construct a simple, practical, low-cost prototype system at last.