Action Recognition Method Based On Motion Vector Prediction

Nowadays,the amount of video increases rapidly,taking more than 90 percents of Internet traffic.So it has been a hotspot in research area to make the computer use these data more efficiently.Besides,among these numerous videos,videos that focus on human behavior occupy the vast majority.Thereore,it is important to explore the method of the recognition of human behavior in video.Most video behavior recognition methods can be divided into the following four categories: dual-stream method based on temporal and spatial features,convolution method based on 3D convolution,LSTM method based on multi-frame image input,and pose estimation algorithm based on extracting human skeleton information.Among them,the dual-stream method has been receiving much attention since introduction.The classic TSN(Time Domain Segmentation Network),one of the dual-stream method,is the basic architecture of many new networks due to its simple and clear network structure and high classification accuracy.However,the manual design of optical flow characteristics required by the dual-stream network is cumbersome and time-consuming,and has high storage requirements.Therefore,some people proposed a video behavior recognition method based on the compressed video,which directly reads motion vectors from the code stream as time-domain features.However,even though the speed is greatly improved,it is not as accurate as the optical flow as the time-domain motion feature since the motion vector contains a lot of noise.Therefore,this paper proposes a behavior recognition method based on motion vectors,including following steps.First,the model structure and experimental results of the TSN network are reproduced and tested.Then,an analog optical flow network is used to implement a sub-network for extracting motion-like vectors.After that,the sub-network is embedded in the TSN network to research and test the accuracy of network behavior recognition under different hyperparameters,different scale fusion methods,different optical flow and motion vector fusion technologies.Finally,according to my experiments,better results can be obtained compared with that of optical flow and motion vector only behavior recognition methods,proving that multi-scale fusion of motion vectors and optical flow is a better time-domain motion feature and contribute to network behavior recognition.