Spatiotemporal Feature Learning For Video Action Recognition

Video action recognition is a popular research topic in computer vision,which aims to recognize the actions of people in videos.In traditional image recognition,the spatial information of images,including texture,shape,context and so on,is the key feature for algorithms,which has a great impact on the classification result.However,in video action recognition,the dynamic temporal information such as motion information and long-term dependency also plays an important role apart from spatial information of static frames.Traditional action recognition algorithms usually extract spatial feature from video frames,or extract temporal feature separately using additional algorithm.After that,the classifier generates predictions for those separated features of each frames,and then aggregates the classification score to make a video-level prediction.However,spatial information and temporal information are usually inseparable in video.Extracting spatial and temporal feature disjunctively can not fuse the information of these two domain appropriately,leading to a inferior recognition result.Therefore,according to these observation,we propose a spatial-temporal feature learning method based on deep learning approach where convolutional neural network and recurrent neural network are aggregated in a novel way,allowing the network to learn spatial-temporal feature synchronously in the forward computation stage.By using residual network and convolutional long short-term memory network,we design a recurrent residual network,and improve the performance of action recognition in standard datasets.By adopting experiments on different network structures,we verify the importance of spatiotemporal feature learning in video action recognition.