Research And Application Of Video Behavior Recognition Based On Spatio-temporal Attention

In the field of video behavior recognition,the existing algorithms most focus on the temporal characteristics of video,so as to reduce the time redundancy,and to concentrate computing resources on the key frames of the video.These algorithms ignore the spatial redundancy of the video.The label is playing basketball,and only a portion of the area contains the action of playing basketball,while the rest is the background.We only select the effective areas for subsequent processing.The classification effect of different regions on behavior in video frames is different.Focusing on key regions will help improve recognition accuracy.In this thesis,we uses reinforcement learning methods to design a video behavior recognition algorithm based on spatiotemporal attention;at the same time,a fully automatic intelligent video behavior recognition monitoring system is developed,which will the advanced video behavior recognition algorithm is embedded in the system to detect abnormal human behavior and issue an alarm.The specific work is as follows:1.A video keyframe extraction algorithm based on reinforcement learning method was proposed to remove time redundancy.The algorithm uses the Mobile Net V3 network to extract the features of video frames.We designed an algorithm to calculate the action values of video frames,which measures which frames in the video have significant impact on the behaviour recognition.We keep these keyframes and remove those unimportant frames;2.Design a video action recognition algorithm based on spatio-temporal attention,we named it TSAN.The network is divided into two parts: a temporal feature extraction module and a spatial feature extraction module.The temporal feature extraction uses the Mobile Net V3 network to extract features,and the summary network is used as the backbone network.The summary network is a recurrent neural network.The summary network captures temporal features and is based on whether the action in the video frame affects the video behavior recognition to select the video frame,extract the key frame,filter out the non-key frame,and remove the time redundancy.The key frame is input into the policy network π,which can extract an image block patch that removes spatial redundancy from each key frame.In order to capture the feature information in the image block more effectively,we use a high-capacity,igh-precision Res Net50,so that the motion features of video actions can be effectively extracted,and finally accurate detection results can be obtained;3.Combined with the designed video behavior recognition algorithm,a set of fully automatic intelligent video behavior recognition monitoring system is built,which can perform behavior recognition detection and abnormal behavior alarm on the content of the surveillance video.When an alarm event occurs,it can automatically record the alarm that occurred information and save related videos.