Research On Action Recognition Method Based On Graph Convolutional Neural Network

As a popular research area of computer vision,human action recognition has been widely applied in human-computer interaction,abnormal action detection and intelligent security scenarios in recent years.Although a lot of research studies exist in this field,improving the accuracy of skeleton-based action recognition is still an urgent research problem to be solved.Human actions are composed of different body languages in different scenes,which can be abstractly described as a serial of skeleton data.For a set of skeletons of simple actions,it has high discrimination features.But for actions with similar skeletons,it remains a challenge to achieve high accuracy and error-free recognition.In order to improve the recognition rate of actions with similar skeleton data,the thesis proposed gated graph convolutional network based on multi-stream fusion for human skeleton-based action recognition.In the construction of the skeleton graph,a weakly dependent spatiotemporal graph is proposed to enhance the global representation of nodes;To address the limitations of the inherent skeleton map and improve the generalization ability of the model,the thesis introduced the adaptive adjacency matrix and gated graph convolution module;To enhance the recognition rate of similar skeletal actions,an attention mechanism is designed based on motion information,and calculates the fluctuation range of key points on time series and the rotation amplitude of corresponding bone joints through data pre-processing to achieve the effective differentiation of confusing actions.Furthermore,the paper constructed a custom dataset for medical action scenarios,which contains 12 actions of personal protective equipment donning and doffing named PPE-DD.In conclusion,data preprocessing and ablation comparison experiments are conducted on three datasets for the proposed method in the thesis,respectively.The experimental results show that our method has higher accuracy and stronger generalization for skeleton similar actions.In order to solve the problem of homogeneity of the skeleton data in the graph convolutional model,the thesis proposed an action recognition method based on multimodal fusion graph convolutional network.On the fusion of modalities,the thesis proposed to fuse RGB features in the video to enhance the closeness of skeleton modalities to the action background;and secondly,the thesis also proposed to fuse sensor modal information such as acceleration and gyroscope to improve the extraction capability of human motion features.In terms of fusion approach,the thesis designs a fusion method in spatial dimension,which means introducing the fusion of new modal nodes with the original skeleton nodes to improve the discrimination of spatial features.Furthermore,the thesis designs a fusion method in channel dimension,which means extracting the local feature information of RGB with the features of sensors attached to the attributes of graph nodes to improve the global graph node representation capability.To verify the effect of fusion with different modalities,detailed ablation comparison experimental analysis is conducted on two public datasets with multimodal.The experimental results show that the fusion of either RGB modality or sensor modality can improve the accuracy of action recognition and the robustness of the model.