Research On Human Pose Estimation With Monocular Videos For HCI Applications

Automatically analyzing and understanding human motion has been an important field of computer vision research for many years. The interests are inspired by not only human curiosity of exploring and imitating ourselves via computer but also the large potential market growing with the prevalence of personal computers and consume electronics. This thesis focuses on the problem of 3D human pose estimation with monocular camera for novel human computer interaction (CHI).Monocular 3D human pose estimation is one of the most challenging topics in computer vision. The difficulties lie in both the input and the output. The observation of the system is always complicated natural image, while the system state within a high-dimensional space. Inference from the observation to the state is essentially a nonlinear dynamic process. Moreover, a monocular 3D human pose estimation system has to be accurate, robust and real-time for CHI applications and the system initialization procedure should involve users as less as possible. With these requirements, we'have designed algorithms for all modules of a monocular 3D human pose estimation system and integrated them into a CHI prototype system; therefore, a CHI system based on monocular 3D human pose estimation is implemented.In this work, we define three key technologies for monocular 3D human pose estimation: image feature extraction, human pose estimation and automatically initialization. Our research on image feature extraction targets commonly-used low-end cameras, such as web-cameras. We adopt HSV color space, which is consistent with human visual system, to improve the effectiveness and robustness of image feature extraction. As far as the human pose estimation is concerned, we propose a hybrid model, combining discriminative model and generative model, to estimating 3D pose. The algorithm firstly locates a local subspace of human pose by a discriminative model, and then refines the pose within the local subspace by a generative model. In this way, the model takes on advantages of both models. As to automatic initialization, we focus on semi-automatic video object segmentation and evaluation metrics. An efficient tool for video object segmentation could help users provide training data easily and consequently reduce users' manual work during initialization.Based on all the proposed algorithms, we develop a novel CHI system based human body movement. To further evaluate the CHI system, a web-camera based video game is implemented, which could be used for interaction design. Based on this game, we carry out a user study and discuss the problems and opportunities for the novel CHI system. The result of user study demonstrates the effectiveness of the proposed monocular 3D human pose estimation system, meanwhile shows us the attractiveness and brilliant future of the novel CHI system based on human movement...