Action Proposal And Activity Recognition Based On Attention LSTM

The purpose of activity analysis is to detect and identify ongoing activity in video so it can enable the computer system to understand the activity and describe the scene further more.However,behind the massive video data is the uneven content,which brings unprecedented challenges and pressure to video activity analysis undoubtedly.Although various existing analyzing models can effectively analyze and identify the activity in video,there are still some limitations: The model of activity analysis is mostly limited to the low level features and it is difficult to express the specific process of activity;Complex background cluster and variation in illumination conditions make the video contain a large amount of background redundancy information;The different length of the video makes it contains a large number of redundant frames that are less relevant to activity analysis.In this regard,this thesis analyzes the basic features of activity tasks combined with the information dependence characteristics of recurrent neural networks,introduces spatial-temporal attention machanism in Long Short-Term Memory network so it can explore spatial-temporal context information and the activity expression process and extracts salient area of key frames from videos.The valid information can enhance activity expression.Considering the above problems,the main work of this thesis is as follows:(1)In view of the fact that most existing activity analysis still contains a lot of noise and can not understand the expression process of activity about the cognitive perspective,this thesis introduces attention mechanism in Long Short-Term Memory network to explore spatial-temporal context clues about activity and pay attention to spatial-temporal effective information to improve the efficiency of activity analysis.(2)As for most of the current action proposal methods are inefficient and cumbersome.This thesis proposes action proposal method based on spatial attention which explores important stimuli and suppresses unimportant background noises in the scene.The training process only requires the category information of action and does not need true bounding box of action so it can enhance the efficiency of action proposal.(3)Aiming at the fact that the video contains a large amount of background cluster and in order to express the activity more accurately,we also use two-stream network to explore the detailed motion information by the temporal motion features except the appearance features.Moreover,due to the different durations of the video and differentregions of activity occurrence,the proposed spatial-temporal attention mechanism can effectively extract the salient region of the key frame and reduce the interference of the redundant information in the video to the activity recognition.