Video Based 3D Human Body Reconstruction And Motion Capture

With the fast development of digital multimedia technology, the requirement to motion capture technique is increasing rapidly, especially in industries of digital games, movies and animation application. At present, the cost of commercial motion capture system is very expensive. Further more, they have many disadvantages, such as over-strict environment requirement, special hardware attached to actor's body and so on. Based on techniques of computer vision and graphic, we present some new idea and improvement. A 3D human body reconstruction and motion capture method is proposed, which is novel, practical and low cost.Firstly, an iterative method which performs camera calibration and 3D feature point reconstruction simultaneously is proposed. It needs very little human interaction. After convergence, the camera's parameters and reconstructed characteristic points are obtained accurately.Then we make use of sparse points in collaboration with the generic 3D model to complete the 3D model deformation via interpolation by radial basis function (RBF). This method can be applied to subject with different shapes and sizes. But due to the fact that sparse feature points do not contain enough information, the limbs of the subjects are not proper fitted.Thirdly, to improve the results of the 3D reconstruction from feature points, we present a silhouette matching method which match and register the model silhouette to subject's silhouette in the real images. The registered information is then used to formulate the RBF deformation vectors for transforming the shin to fit the target subject. The results gave an average error of less than one pixel.Fourthly, despite of the fit silhouette, there are still some local distortions on the geometry surface because of the non perfect camera calibration and the low image definition. To resolve these problems, three sorts of filters are designed which are smoothing filters, slice filters and neighbor triangle normal filters.Fifthly, due to the fact that the illuminations in the images are very different, the textures extracted directly from images are not consistent. To deal with this problem, we slice the reconstructed model by using a rational cutting plane. After slicing the model, synthesis texture is obtained and mapped onto the model surface. Finally, to achieve marker-less human motion capture which using the cluttered background video as input, an analysis-by-synthesis method is proposed. The method estimates the matching error between the synthesized images and the real images by comparing the all pixels. Then the downhill simplex simulation annealing algorithm is adopted to minimize the matching error. Experiment results indicate that this method is accurate and stable.