Research On Action Recognition Based On Spatiotemporal Graph Convolutional Network And Its Application In Drama Character Performance

Chinese opera has a long history and a wide variety of operas.According to incomplete statistics,there are about 360 kinds of operas in various regions of China,and tens of thousands of traditional operas.The popular and famous operas are: Peking Opera,Henan opera,pingju,Kunju,Shaoxing Opera,Huangmei Opera and so on.And qinqiang as one of them,it is.Qinqiang opera is not only an important part of Chinese traditional culture,but also an important non-renewable spiritual and non-material wealth of Chinese civilization.As Qinqiang opera is limited by regional factors,and as time goes by more people,especially the younger generation,are no longer interested in the traditional folk performance of Qinqiang opera.In order to inherit and protect the opera culture of Qinqiang opera,modern technology can be combined with Qinqiang performing arts,such as virtual reality technology,to realize the diversification of Qinqiang performing forms.The action characteristics of the characters in the Qinqiang opera performance are complicated.How to make the computer accurately identify them is an urgent problem to be solved.Because human skeleton is robust and not affected by background noise,many researchers combine it with graph convolutional neural network for motion recognition.Due to the temporal nature of skeleton data,it is a great challenge to extract its temporal and spatial features,and the existing methods tend to ignore the dimension of channels.In addition,these methods are only limited to the vertices corresponding to the same joint between frames,making it too simple to extract the motion features of each node between frames.Moreover,these methods only make use of first-order information such as skeleton node,which can better express the motion features of bones.In view of the above deficiencies,this thesis,based on Spatial Temporal Graph Convolutional Networks(STGCN),finally proposes a multi-stream extended space-time graph attention model to improve the recognition rate of characters’ actions in Qinqiang opera,and the specific contents are as follows:(1)In view of the difficulty in extracting spatial temporal features,this thesis proposes a model of Spatial Temporal Graph Convolution Network combined with attention mechanism,which introduces polygraph attention and channel attention mechanism to improve feature extraction.The graph attention mechanism can obtain the respective attention coefficients of different neighbor nodes to the central node,which can help the model learn structural information.Channel attention mechanism can make the model focus on important channel features,and the two mechanisms can be combined to improve the effect.The X-Sub and XView of the model on NTU-RGB +D data set improved by 2.7% and 3.6%,respectively,compared with the benchmark.top-1 and top-5 accuracy were 3.1% and 2.9% higher than the baseline in Kinetics dataset,respectively.In addition,the proposed model was compared with the self-made Qinqiang data set,and the recognition accuracy of six kinds of movements "spitting fire","tea mouth pot","bow and arrow","top lamp","mounting horse" and "sitting horse" were all higher than ST-GCN.(2)In view of the existing methods can’t extract the same joint point and its neighbor nodes information between frames,and for the second order information,such as bone,etc.)the use of fewer problems,this thesis proposes a flow based on the expansion of space-time figure attention network model,the model first introduced time extension module for more adjacent joints between frames corresponding vertex add edge sampling area to expand the time dimension.Then,the multi-flow network framework is used to integrate the node,skeleton and their respective motion information to improve the model performance.The accuracy of the model in the NTU-RGB+D dataset was 5.3% and 5.6% higher than the benchmark in X-Sub and X-View,and 4.8% and 4.6% higher than the benchmark in top-1and top-5 in Kinetics dataset,respectively.At the same time,in the self-made Qinqiang data set,the accuracy of "spitting fire","tea mouth pot","bow and arrow" and "mounted horse" movements are all above 90%,and the recognition rate of "headlamp" and "sitting horse" movements are also close to 90%.(3)Design and implement the action recognition system of Qinqiang opera characters,which can realize the data preprocessing of opera characters,Open Pose extraction of key points and action classification,and realize the combination of algorithm and practice.