Research On Human Action Recognition Based On RGB-D Image Sequences

Human action recognition is a subject with significant academic value and extensive application value in computer vision.Traditional action recognition methods based on RGB image sequences are affected by light and background.With the development of depth cameras,some recent literatures have researched on recognizing human actions in RGB-D image sequences.This paper proposed a novel action recognition method based on multi-feature fusion.In this method,spatial-temporal features and depth features are merged in a random forest framework.The human body joint coordinates obtained from depth image sequences are processed into two new depth features: displacement feature and part-center feature.The displacement feature is used to capture the relative motions between joints and describe three-dimensional structure changes.We group the human body joints into five body parts according to the natural structure of human.Finding a central point for each part and then the weight of the body part is obtained by calculating the distance between the part center and the body center;The keyframes are chosen from depth image sequences.The part-center feature is the difference of the weight from the same part in two sequential adjacent keyframes.We use this part-center feature to capture the sequential motion information of the five part-centers.The spatial-temporal feature is extracted from RGB image sequences to describe the human motion information and the apparent information.We sample dense trajectories firstly and utilize foreground detection approach to eliminate the effect of complex background.Then spatial-temporal features are constructed by Bag-of-Words model with trajectories from foreground.Finally,the random forest framework is proposed to fuse both spatial-temporal features and depth features effectively.Experimental results on MSR Daily Activity 3D dataset and MSR Action3 D dataset demonstrate the effectiveness of the proposed method.