An Action Recognition Method Using Improved Hidden Conditional Random Field Model

Having the broad application prospects and the potential social and economic benefits, human action recognition has become one of the leading directions in the field of computer vision, which has attracted many domestic and foreign researchers to conduct in-depth research and obtain significant research achievements. However, the contradiction between the video feature extraction computing complexity and action recognition accuracy still has not been solved effectively. Human action is a dynamic process, which changes continuously over time and also has a certain periodicity. The resulting data stream will have many similar observation sequences at various temporal resolutions, thus, capturing discriminative information from a single learning model may prove to be difficult. Although the hierarchical learning model can achieve better high-level abstract feature representations, the modeling process is too complicated. Based on summarizing and analyzing human behavior recognition methods commonly used in the domestic and overseas, this thesis has done the following work for the above problems.1. This thesis summarizes the moving object detection methods commonly used in the domestic and overseas, analyzes the advantages and disadvantages of various moving object detection methods, and proposes a novel feature extraction method: 1) Modeling the original input video through Gaussian mixture model to extract the foreground binary image of moving object, and calculate the centroid coordinates of the foreground binary image. 2) Locating the image foreground block in original video frame, which is corresponding with the foreground binary image, by centering around the centroid coordinates. Then, using 3 layer structure of pyramid to extract the pyramid of histograms of orientation gradients(PHOG) feature from the image foreground block as the feature vector of the original video frame. 3) When the same thing has been done for the whole original video, the corresponding temporal observation sequence can be obtained. The feature extraction method above significantly reduces the interference caused by background edge information in the process of image feature representations.2. This thesis models temporal observation sequence by using hierarchical sequence summarization hidden conditional random field(HSSHCRF) model, capture thecomplex spatio-temporal dynamic information of temporal observation sequence through a hierarchical sequence summarization approach. Namely, this method builds up a hierarchy dynamically and recursively by alternating sequence learning and sequence summarization. For sequence learning, this method uses hidden conditional random field(HCRF) model to learn hidden spatiotemporal dynamics; for sequence summarization, the observations that have similar semantic meaning in the latent space are grouped together. For each layer, this method learns an abstract feature representation through non-linear gate functions. This procedure is repeated to obtain a hierarchical sequence summary representation. This method preserves details in low-variance groups while improving the discrimination ability of high-variance groups, trains model through an efficient learning method and shows that the complexity of the model grows sub-linearly with the size of the hierarchy.This thesis verifies this action recognition method in a classic action recognition video database, while analyzing and comparing the experimental results, which show that this method effectively relieves the contradiction between the video feature extraction computing complexity and action recognition accuracy.