Action Recogniton Based On Deep Neural Networks With Visual Attention Mechanism

As one of the most important research field of computer vision,action recognition has a lot of application requirements.However,due to the fact that video data consists of hundreds of images as well as temporal information,it is quite a big challenge for video analysis to extract informative features of videos quickly and accurately.In this paper,deep learning technologies and visual attention mechanism are both applied to human action recognition.Deep learning mainly depends on deep neural networks,which simulates activities of human brains and find the pattern of real-world data just like humans do.The advantage of using deep learning is that it never requires hand-craft features.While the main idea of visual attention is that human eyes will focus on certain areas when watching videos,rather than the whole frame,so such a model is believed to be suitable for analyzing data that changes dynamically.In this paper,not only the structure of deep neural network gets improved but also a new framework for action recognition is proposed.Convolutional neural networks are used for video feature extraction,and LSTM neural network is used to model the video data,and the visual attention mechanism is implemented by LSTM units,which enable the model to memorize as well as infer during the training process.The model would learn to attend like humans.Experiment results on ucf11 and hmdb51 benchmark datasets achieved more than 89% and 82% accuracy rate,which is far beyond the conventional video analysis algorithm.The paper also verified that visual attention greatly benefits the deep neural networks.