Research On Human Motion Analysis From Monocular Videos

Human motion analysis from monocular videos is an important and fundamental research topic in computer vision. The principal mission is to automatically capture human motion data from visual observations. As a new research subject in visual understanding domain, human motion analysis is built on the achievements of many existing research subjects. It has wide range of applications including virtual reality, intelligent surveillance, human computer interaction, sport performance analysis and so forth. In this thesis, we mainly studied several key technologies of human motion analysis, such as visual feature detection and representation, depth ambiguity in discriminative methods, computational complexity caused by the high dimensional pose state space and the designing of optimization strategy.Regarding these issues, this paper engages in the following work:(1) We proposed an example-based human motion analysis method. The spatio-temporal structure is used to reduce the error estimation rate caused by depth ambiguity. We proposed a motion detection method for human silhouette detection which combines the frame difference and background subtraction. The proposed motion detection method can be used for silhouette detection effectively. We use shape context for silhouette matching, which is used for pose retrieval. We proposed an example-based human analysis method. Based on the relationship and distribution of motion state space, the spatio-temporal information is extracted by probability modeling and statistical modeling. The spatio-temporal information is introduced in to the process of human analysis which reduces the error estimation rate caused by depth ambiguity.(2) We proposed a latent space learning method for human motion analysis, which makes motion analysis more efficient and accurate. We use motion capture data to learn the latent space of human motion, by which the aim of both reducing dimensionality and extracting the prior knowledge of human motion are achieved simultaneously. As the latent space is low dimensional and contents the prior knowledge of human motion, it makes motion analysis more efficient and accurate. Then we proposed the framework for motion analysis in latent pose state space. Evolutionary computation methods are applied for pose optimization. We designed the pose estimation method. And sequential evolutionary computation method is proposed for motion tracking. Experimental results demonstrate that the framework makes motion analysis more efficient and accurate, which provides a general framework for motion analysis in latent space.(3) We proposed an Annealing Particle Swarm Optimization (APSO) based method for human motion analysis, which makes pose optimization has a large probability to be global optimum. As APSO utilize the constraints and structure of the latent space, it reduces the search space and makes pose optimization to be more efficient. We set the inertia weight parameter in the particle velocity update equation to be a decreasing function, which guides the swarm from a global to more local exploration. We introduced simulated annealing (SA) into particle swarm optimization. The concept of SA is that the particles with low fitness are also accepted with a probability to further increase the search space. Experimental results show that APSO can get global optimum solutions more effectively.(4) We proposed an Immunity Genetic Algorithm (IGA) based method for human motion analysis. The utilization of constraints of latent space makes motion analysis to be more accurate. The concept of IGA is to use the prior knowledge of human motion to suppress the phenomenon of "degradation" in the optimization process. The core of the method is the genetic operators, vaccine designing and immune operators. We defined a set of genetic operators for population evolution. Two vaccines are designed based on the constraints and structure of the latent space. We use vaccination and immune selection steps to apply the prior knowledge into the process of optimization. The experimental results show that:IGA has good convergence and optimization capabilities which improved the accuracy of the motion analysis results.(5) We proposed a Clonal Selection Algorithm (CSA) based method for human motion analysis. The affinities of antibodies are increased by the "Clone-Selection" mechanism, which improved the local optimization ability of motion analysis method. "Clone-Selection" mechanism of CSA explores local areas around antibodies by introducing blind perturbation, which increased the affinity of antibodies. So CSA has good local optimization capability, which improved the accuracy of the motion analysis results. CSA substitute some individuals of the population by some randomly generated individuals to maintain the diversity of the population, which ensure the ability of global optimization. Experimental results show that CSA has good optimizing performance, which can achieve stable human motion analysis.(6) We proposed a sketch based 3D human pose modeling method for motion editing, which meet the basic requirements of human animation from videos. Sketch is introduced for human pose modeling. The human pose model is built based on sketch understanding and human template deformation. As the model is a data-driven deformable model, it can be used for human animation directly. We propose a pose model editing method, which enables the users to edit the pose of human model by 2D sketch interaction. Combined with motion smooth method, the pose editing method can be used for animation editing. Experimental results demonstrate our pose modeling method meet the basic requirements of human animation.