Human Action Recognition Based On Skeleton Information

Human action recognition is a hot topic in the field of computer vision.Its main purpose is to correctly classify the human actions in the video.This technology can be applied to intelligent video surveillance,human-computer natural interaction,sports video analysis and self-driving areas.However,how to construct effective features to describe the human actions in the video has always been a very challenging issue.Through deeply exploring the human skeleton information,we propose joints kinetic and relational features.This set of features consists of 4 major categories of features and 36 kinds of sub-features:1.The kinetic features are the combination of velocity,acceleration,angular veloc-ity,angular acceleration,speed,acceleration rate,kinetic energy,kinetic energy change,potential energy,potential energy change,total energy,total energy change and normal-ized positions.There are 13 kinds of sub-features in total.From the aspect of the motion change and the energy change of the joints,this group of features fully mines the kinetic information of the human skeleton.2.The correlation relational features are the combination of velocity correlation,acceleration correlation,angular velocity correlation,angular acceleration correlation and energy flow.There are totally 5 kinds of sub-features.This group of features describes the motion correlation relations and the energy change correlation relations between each pair of joints.3.The distance relational features are the combination of horizontal distance re-lation and its trajectory,vertical distance relation and its trajectory,orientation sine relation and its trajectory,orientation cosine relation and its trajectory,eigen vector di-rection distance relation and its trajectory,link distance and its trajectory.There are 12 kinds of sub-features in total.This group of features describes the distance relations in particular directions between each pair of joints.4.The geometric relational features are the combination of the joint vector inner product and its trajectory,joint vector cosine similarity and its trajectory,area perimeter rate and its trajectory.There are 6 kinds of sub-features in total.This group of features describes the geometric relations among each triplet of joints.These features are combined as joints kinetic and relational features.This paper makes a comprehensive comparison of the sub-features.This set of features has yield-ed good results on the JHMDB dataset,the sub-JHMDB dataset and the Penn Action dataset.In addition,as each part of the action recognition system will have a certain impact on the final recognition results,this paper explores the framework of the action recognition algorithm to find which one is suitable for the joints kinetic and relational features.The most suitable bag of visual word model is based on K-means clustering and vector quantization.The most effective classification model is the multi-channel RBF-?2 kernel support vector machine.In general,by fully mining the skeleton information,this paper presents a set of joints kinetic and relational features,and explores which bag of visual words model and classification model is suitable for this set of features.We proved this group of features is effective through experiments.This paper also provides suggestions to do research on human action recognition based on skeleton information.