A Cuboid CNN Model With An Attention Mechanism For Skeleton-Based Action Recognition

The human three-dimensional skeletons data information obtained from the depth sensor,due to its small data and relatively simple data form,can fully express the action information of the human body,which leads to an in-depth study of the action recognition.Although many existing action recognition methods have achieved good recognition performances,there are still many problems exposed.First,most of the existing action recognition methods usually consider human actions from a global perspective,while local parts of the human limbs play an important role in predicting human action.Second,the changes of human actions are the result of a series of joints interactions,the joints in the process of human action change are related.In the process of extracting features,if all joints of the whole human body are given fixed weights,the recognition accuracy will be reduced.If the correlation between joints is not considered,each joint will cause a scatter structure,which is not conducive to the training of the depth model.Finally,most behavior recognition methods consider the training of action recognition using distance or position information as features,but human actions are usually directional.All of these factors led to the failure of action recognition.Based on above:(1)This paper proposes a cuboid arranging strategy to organize human body joints information.First,the changes of human actions,usually due to changes in the relative distance between joints.We use the distance and direction information between the key joints of the human body to express human behavior;secondly,the cuboid arranging strategy will organize the information of human joints,so the correlation between joints is enhanced accordingly.Our cuboid feature representation is also used in multi-person behavior recognition.(2)Considering the attention mechanism of human vision,we propose a novel attention mechanism to enhance the salient information for the cuboid features,and to mine the salient features that lead to behavioral changes.The cuboid feature associate with attention mechanism will be capable to suppress useless noise information while enhancing the salient features.(3)Existing action recognition datasets do not address the multi-model and multi-view problem.Furthermore,the background of actions in real-world applications is very complicated.To investigate these problems,this paper collected the CAS-YNU MHAD action recognition dataset.The dataset fully considers the perspective,age,gender,and collection scenarios,making up for the lack of background conditions for many RGBD datasets.Finally,we validate the effectiveness of our proposed strategy on three popular datasets NTU RGB+D,UTD-MHAD,UTKinect datasets and CAS-YNU MHAD dataset.The experimental results show that the proposed method achieves the latest recognition performance.