Group Activity Recognition Based On Spatio-temporal Information

The research of group activity recognition has become a hot spot in the field of computer vision.It has broad academic prospects and immeasurable actor value in intelligent control,unmanned driving,virtual reality.Group activity recognition focuses on the group of actors in the scene.Due to the confusion of background caused by occlusion and misalignment in video capture,as well as the complex relationship between actors,it has become a challenging research.Existing group activity recognition methods have achieved certain results,they pay attention to the information of the relationship between actors and the group they belong to.Howere,they still ignore spatio-temporal information at the actor level,group level and frame level.In order to solve the above problems,this thesis focuses on how to get accurate extraction of spatio-temporal information,so two group activity recognition methods that pay attention to spatio-temporal information are proposed.In the video frame,there may be obstacles covering the target actor,and some spatio-temporal information may be ignored.Therefore,it is not possible to rely solely on target tracking and target detection for feature extraction.Therefore,this thesis proposes a spatio-temporal attention mechanism,which get the key action information in the video frame according to the obtained temporal attention score,and the spatial attention module get the correlation information between different pixels according to the obtained spatial attention score,so as to get the key frame and key activity area information.The above methods was verified on the two more authoritative datasets,namely the volleyball dataset and the collective activity dataset.The results directly proved that the group activity recognition model based on the spatio-temporal attention mechanism can improve the accuracy.In view of the fact that the current activity recognition methods ignore the spatio-temporal correlation between actors,an effective group activity recognition model based on spatio-temporal relation networks is proposed to capture potential spatio-temporal characteristics in an end-to-end manner.Firstly,this thesis proposes a spatio-temporal relation module that get the feature correlation between feature nodes from the time dimension and the space dimension.Secondly,actor spatio-temporal feature module and multi-actors relation module are designed to extract spatio-temporal semantic information and relation features between actors,and then perform feature fusion,and perform activity classification.Finally,the proposed method is experimentally demonstrated on the two commonly used and challenging datasets,the volleyball dataset and the collective activity dataset.The results on these two datasets show the excellence of the method in this thesis.