Recognition of Human Actions based on 3D Pose Estimation via Monocular Video Sequences

We propose a system to recognize both isolated and continuous human actions, from monocular video sequences, based on 3D human pose estimation and cyclic hidden Markov models (CHMMs). First, for each frame in a monocular video sequence, a 3D human pose estimation scheme is applied to extract the 3D coordinates of joints of the human object with actions of multiple repeated cycles. The 3D coordinates are then converted into a set of geometrical relational features (GRFs) for dimensionality reduction and improved discrimination. For further dimensionality reduction, the k-means clustering is applied to those GRFs to generate clustered feature vectors. These vectors are used to train CHMMs separately for different types of actions based on the Baum-Welch reestimation algorithm. For recognition of continuous actions, which are concatenated from several distinct types of actions, a designed graphical model is used to systematically concatenate different separately trained CHMMs. The accurate estimation of the 3D human poses, the effective use of GRFs and CHMMs significantly improve the performance of both isolated and continuous human action recognition problems.