| The action recognition algorithm has been widely used in many real scenes,such as intelligent monitoring,human-computer interaction,etc.Skeleton data has the advantages of simple data form and low computational cost,so action recognition based on skeleton data is becoming popular.The skeleton-based action recognition models have been gradually applied in actual scenes.Their robustness is a very important and key indicator,but it has not been comprehensively analyzed and verified.Compared with traditional text and image data,skeleton data is vulnerable to disturbance due to its unique topology.On the other hand,some studies have proposed that semantics may help improve the robustness of the deep learning models,so how to effectively analyze the robustness of the models and verify the validity of semantics is the basis of the research,and how to effectively improve the robustness of the skeletonbased action recognition models by combining semantics is the key problem to be solved finally.In response to the above problems,this study first designs a robustness verification analysis method for skeleton-based action recognition models based on multi-dimensional features,fuses dynamic features and emotional features,generates disturbances by alternating direction multiplier method,and analyzes and discusses the performance differences between different models from the two dimensions of misclassification and insensibility.The results show that the method can generate effective disturbances that are more close to the original skeleton samples.It achieves 100%misclassification rate on different models,and also verifies that semantics can effectively improve the robustness of the models.Then,from the perspective of model structure,a highly robust skeleton-based action recognition model based on semantic-guided is proposed,which integrates temporal and spatial semantics,uses attention mechanism to assign different weights to different frames and different joints,and proposes an adaptive computing graph structure in the graph stack layer to flexibly apply to different samples.The experimental results confirm the effectiveness of attention mechanism and adaptive graph convolution.At the same time,compared with other classical models,the model achieves 95.3%accuracy with 1.13G parameters,achieves superior performance with lower parameters,and also verifies its high robustness over other models.Finally,a skeleton disturbance correction model based on semantics is proposed from the perspective of additional network,which can be used as the external wrapper of skeleton-based action recognition models,detect and correct the disturbance contained in the sample,and improve the resistance of the model to abnormal data without changing the existing model structure and parameters.The experimental results show that the detection and correction accuracy of the correction model can reach more than 90%in different attack methods,and it is universal for different models and different attack methods. |
