Human Action Recognition Research Based On Key Frame And Primitive

As one of the most popular topics in computer vision, human action recognition plays a significant role in many areas such as human machine interaction, video surveillance etc. Generally speaking, action recognition includes three parts:feature extraction, action representation and action classification. Due to the nature of intra-class variation and inter-class similarity of action, current research on action recognition mainly focuses on how to extract robust features and how to represent action with these features. As a result, this thesis propose a method for feature extraction based on motion block and two algorithms for action representation and classification which include key frame-based action representation and classification, primitive-based action representation and classification. These two algorithms achieved excellent performance on two public datasets:KTH and UCF sports, both proved to be very practical and effective.In order to represent the motion nature of action and resist occlusion, a new method for motion block extraction is proposed in this paper with graph-based clustering from the perspective of human bounding box detection. More robust motion blocks are further selected via building histogram of optical flow inside and computing entropy. In order to resist the effect of action caused by scale change, the relative distance of gravity center between motion block and bounding box. The orientation of motion block is added for the motion feature of action. Meanwhile, histogram of gradient of each frame in video are extracted according to human bounding box which represent the shape nature of action.Research indicates that human beings can recognize action according to several representative frames, namely key frames. A new method of key frame extraction is then proposed in this paper based on motion blocks. Key frames are extracted from the huge video stream according to the detection of abrupt change of motion in action which then reduce the computation of the incoming feature extraction. In order to represent the motion and shape nature of action sufficiently, motion and shape descriptors are extracted based on Gaussian mixture model and bag of words with the feature of motion block and histogram of gradient. Actions are classified by combining these two descriptors linearly and the use of Nearest Neighbor classifier.Although key frame-based action representation and classification reduces the computation of feature extraction, it is volatile to the noise of abrupt motion change. What’s worse, the performance of motion and shape descriptors, which are based on Gaussian mixture model and bag of words are very sensitive to initial parameters. The inter-class similarity is also ignored when representing action. Aiming at these defects and considering that action consists of a series of sub-actions namely primitives, a new method for primitive extraction is proposed based on hierarchical clustering which can determine the number of primitives adaptively and eliminate the redundancy among actions. In order to enhance the robustness, the statistical feature of primitive is represented based on Gaussian mixture model. With the feature of motion block and histogram of gradient, motion and shape primitives are obtained. Then action is represented by primitive sequence in order to keep the temporal relation among sub-actions. In the end, actions are classified via sequence matching.