Research On Action Recognition Algorithm Of Plague Prevention Based On Multi-stream Network

The new crown epidemic has ravaged the world and brought huge losses to the world economy and people’s lives.According to estimates by WHO experts,humans may have to coexist with the new coronavirus for a long time,and personal protection is the most effective way to prevent the infection of the new coronavirus.Therefore,correct and timely personal epidemic prevention measures are particularly important.Personal protective measures mainly include: correct hand washing methods,wearing masks,and not blowing your nose.Identifying whether these protective measures are in place can help reduce the spread of epidemic infections.A multi-stream network refers to a network that inputs data streams of multiple modalities into different branch networks and uses a multi-stream fusion mechanism for fusion,so as to take advantage of the advantages of different data streams to improve the accuracy and performance of the algorithm.This paper studies a human action recognition algorithm based on a multi-stream network in epidemic prevention and control scenarios and conducts a large number of test experiments.The main work of the thesis includes the following aspects:First,the multi-stream fusion network algorithm is studied.It mainly includes: 1)Convolutional layer operation,weight-copy mechanism,cross-entropy loss algorithm and back propagation algorithm,as well as weight update and optimization algorithm;2)Feature addition,feature splicing and averaging in multi-stream convolutional neural networks,weighted and maximum multi-stream fusion mechanism.Secondly,the human action recognition algorithm based on multi-modality is studied.It mainly includes: 1)3D CNN method and dual-stream method,two action recognition algorithms based on RGB video;2)Depth spatio-temporal interest point method and DMMs method,two action recognition algorithms based on Depth video.The advantages and disadvantages of different modal data and mainstream algorithms are analyzed.Thirdly,the design and implementation of human action recognition algorithm based on multi-stream network is completed.It mainly includes: the action category of the public action recognition dataset,the basic principle analysis of the improved single branch network and the multi-stream network algorithm;The FFmpeg key frame extraction of RGB and Depth data,resizing,pixel value normalization,data enhancement strategy and middle realization of preprocessing such as value filtering;Realization of feature extraction of spatio-temporal domain flow,motion domain flow,time domain flow and HHA depth coding;Parameter migration and training of the improved algorithm models.Finally,the experimental test and result analysis of plague prevention and control action recognition are completed.It mainly includes: constructing the plague prevention and control(color-depth)RGB-Depth action dataset,collecting the action data of 50 people,and doing preprocessing and extracting features.After comparing and analyzing the different training strategies of the improved single-branch network algorithms,the recognition effect of the stream network algorithm under different fusion mechanisms and public datasets shows that the ASTM network algorithm and the DCNN-HHA network algorithm have better recognition results,and the multi-stream network algorithm has the highest recognition rate.Real-time testing is performed with multi-stream network algorithms,and the test results get expectations,indicating that the algorithm can meet the needs of real-time identification and deployment of plague prevention and control behavior,and has certain engineering application value.