Research Of Video Action Recognition Based On Two-stream Information Fusion Network

Recently,video action recognition task has attracted increasing attention due to its applications in security,smart life and robot service.With the development of convolutional neural network,it shows good performance in action recognition task.Different from image-based tasks,video action contains a lot of motion information,which makes it more difficult to obtain high recognition accuracy.Traditional two-stream convolutional networks have a certain improvement on the recognition performance.However,the fusion method of spatial stream and temporal stream needs further research.In this thesis,a novel two-stream information fusion network is proposed to better fuse the spatial information and temporal information and to improve the accuracy of action recognition.Firstly,a temporal information fusion module is designed to fuse the original temporal features extracted from the basic networks into multi-scale temporal features,namely short-time temporal features,mid-time temporal features and long-time temporal features,which can increase the utilization rate of temporal information.Secondly,for the multi-scale temporal features generated by temporal information fusion module,a multi-scale spatiotemporal information fusion module is designed to fuse the different scales of temporal features with spatial features.According to the scale of temporal features,the spatiotemporal fusion is composed of three blocks.In each block,temporal features with the same scale are fused with their corresponding spatial features.We take the spatiotemporal asynchronous fusion method to get different scales of spatiotemporal fusion features from each block.The spatiotemporal fusion features of three scales are integrated to generate a multi-scale spatiotemporal fusion feature for the recognition.Finally,the experiment results on two datasets demonstrate the effectiveness of the two-stream information fusion network.