Group Activity Recognition Research Based On Multi-level LSTM

Behavior recognition is a hot point in computer vision,and development of deep learning has promoted its continuous progress.It has great applications in intelligent monitoring,intelligent transportation,virtual reality and other aspects.Simple behavior recognition refers to the classification of single-person actions,but in practical applications,most scenes are completed by multiple people,such as basketball,video surveillance in shopping malls,etc.Therefore,the group activity jointly completed by multiple people are studied here.Specifically,this article analyzes the group behavior in volleyball videos based on the LSTM network,and analyzes the behavior of each individual and its interactions to complete the group behavior inference with the aim to achieve the group behavior identification.The research difficulty of group behavior recognition lies in the fact that multiple people complete a behavior together.It is necessary not only to analyze the behavior attributes of each individual,but also to consider how to fuse a large number of individual characteristics to represent the group behavior characteristics.The amount of information will inevitably be generated during the implementation process,including redundancy,occlusion,and multiple interactive information,so using only single-person behavior recognition methods to deal with group behavior is not capable.This paper proposes a group behavior recognition model and builds an end-to-end multi-level LSTM group behavior recognition model.The core of the method is to identify the key people in the group and infer the group behavior based on the key people.Although the group behavior is completed by multiple people,the completion of the group behavior is often determined by several core personnels who play a leading role,while others have a very little or no contributions to the group behavior.Individuals who have made great contributions are called "key figures".The key figures are used as the core to establish a model,and their individual characteristics and interaction characteristics are analyzed and inferred to complete group behavior recognition.Based on the above ideas,this paper proposes a group behavior recognition method based on multi-level LSTM(personal level,scene level,group level):First,the bounding box image and scene image of each person tracked are input to CNN for static feature extraction;then they are input to the first-level LSTM network for dynamic feature extraction,and key figures are modeled and ranked according to theaverage optical flow intensity of each individual in a certain period;Then,these ranked individual features are input to the gated fusion unit(GFU: Grated Fusing Unit)according to the order of importance of key people.The GFU uses scene features to locate participants,and fuses the positional relationships between individuals and also between individual and scene,which are regarded as group interaction Information;finally,the group information obtained above is input to the second layer LSTM,and the group behavior recognition is performed using a softmax classifier.Experiments are carried out in volleyball data set and 86.7% recognition accuracy is achieved.