Research On Zero-shot Action Recognition Based On Graph Neural Network

Action recognition is an important research topic in the field of computer vision,which aims to recognize the ongoing action in video.At present,action recognition methods based on deep learning use captured datasets and corresponding labels to train models,and obtain high recognition accuracy on large-scale datasets.However,in practical applications,such action recognition methods are limited due to the difficulty in obtaining the data with rare actions.Therefore,such action recognition methods are limited by practical application.Therefore,it is of practical significance to identify the samples that have never been seen with the condition that the classes of training data are limited.To deal with this problem,researchers have proposed zero-shot action recognition algorithms.These methods are no longer learn the connection between video features and their classes by deep learning.Instead,the models first understand the high-level semantic information of the training data,then transfer the semantic information to the samples that have never been seen,and finally complete the purpose of identifying new actions.The existing zero-shot action recognition algorithms lack to combine specific influencing factors to build knowledge transfer models,which leads to the limitation of the recognition of new actions.To solve the above problems,this thesis analyzes the factors that cause the performance to be limited from several perspectives and proposes new solutions to the problems of existing methods.This thesis integrates deep learning techniques such as knowledge graph and graph neural network to construct new zero-shot action recognition algorithms,aiming to improve the performance of the model by improving the ability of knowledge information to propagate among different actions.The main contents of this thesis are as follows:(1)A zero-shot action recognition algorithm based on self-adjacent knowledge graph and ensemble graph convolutional neural network is proposed.To address the problem that the existing knowledge graph does not accurately represent the similarity between nodes,this thesis designs a self-adjacent knowledge graph to improve the capture ability of visual features of new actions by making actions propagate knowledge information to each other according to the semantic similarity.To further improve the feature transfer ability of the self-adjacent knowledge graph,an ensemble graph convolutional neural network is proposed in this thesis.The network integrates multiple graph convolutional neural networks into a strongly supervised model,aiming to improve the accuracy and stability of constructing visual features for new actions,and thus enhance the final recognition performance.(2)A zero-shot action recognition algorithm based on random mask enhancement is proposed.To improve the knowledge propagation ability of the knowledge graph,this thesis applies the feature enhancement strategy to the knowledge graph and proposes a random mask feature enhancement method,which achieves feature enhancement by randomly erasing the node features.This thesis further proposes a graph neural network based on random mask feature enhancement,which uses a two-branch structure for knowledge propagation of the feature-enhanced mapping and the original input feature of knowledge graph to improve the ability of unseen class nodes to learn visual information and thus improve the algorithm performance.The two zero-shot action recognition algorithms proposed in this thesis are tested in comparison experiments and ablation experiments on the publicly available datasets UCF-101,HMDB-51 and Olympic Sports.The experimental results show that the proposed algorithms in this thesis achieve the state-of-the-art performance.