Research And Application Of Ice Hockey Referee Gesture Recognition Based On Bone And GC-LSTM Neural Network

With the victory of the 2022 Beijing Winter Olympics,the "post-Winter Olympic era" of Chinese youth ice hockey has arrived.Ice hockey is one of the most popular sports in the Winter Olympics,in order to popularize ice hockey knowledge and ice hockey penalty gestures,and make up for the non-professionals’ lack of understanding of the rules of referee gestures,this paper proposes a dynamic gesture recognition of ice hockey referees based on bone and improvement of GC-LSTM.The video was collected to produce the ice hockey referee gesture skeleton dataset,and the bi-feature fusion DFF-GC-LSTM model was used to classify the skeleton-based hockey referee gesture actions.(1)Dataset production: In view of the lack of hockey referee gesture dataset,the text uses a self-made dataset of collecting videos,and uses data crawling to collect professional hockey referee teaching videos on the Internet and divide 28 kinds of gestures.To increase the amount of data,data augmentation is used to extract video frame by frame,and each piece of data is divided into 30 frames per video for timing processing.Use the Media Pipe framework to obtain the key coordinate sequences of hand and pose bones from the video frames and save them as CVS files for use as input to the model.(2)Dual Feature Fusion DFF-GC-LSTM model: The model consists of GCN layer and LSTM layer,which concentrates the key point sequences of hand and pose bones and input them into GCN respectively,extracts the hand and pose map data feature information,and then fuses the two features as LSTM input for time series training,and finally uses the fully connected layer to map the output of LSTM to the hockey referee gesture category label.The model obtains the characteristic information of hands and postures respectively,and can model and recognize the gestures of ice hockey referees at a more fine-grained level.This paper can be realized with only an RGB camera,and it is real-time to capture bone keys using the Media Pipe framework,which supports bone key capture and gesture recognition in real-time video streams.When the dataset is scarce,the use of bone keys has good generalization and can be identified with a small amount of computation.On the ice hockey referee gesture bone dataset,the overall recognition rate of the DFF-GC-LSTM model can reach 91.6%,which is 4.1% higher than the 87.5%recognition rate of the GC-LSTM model.This model can be applied to the teaching and promotion of ice hockey referees and the interpretation of referee actions in ice hockey matches,which has important practical significance and huge potential market.