Human Activity Analysis In Videos

ABSTRACT:Video based human activity analysis aims to detect and recognize human actions in a video by analyzing the content of the video. It has wide potential applications such as intelligent video surveillance, human-computer interface, content based video analysis, etc. Human activity analysis has become one of the most attractive computer vision research fields in recent years.Human activity analysis can be categorized into several groups. According to the number of persons involved in an action, the study of human action analysis includes single person activity analysis, multi-person activity analysis and crowd activity analysis. According to the goal of activity analysis, the task of human activity analysis includes human activity classification and human activity detection. In this paper, we study both the task of activity classification and activity detection. In the activity classification, each video contains only a single person who performs a single kind of activity in both the learning and testing stages. In other words, the start and end frame of the activity in a video is known in advance. The task of activity classification is to classify each video into one of the activity groups. In activity detection, there could be multiple kinds of activities in a video. Both the start and end key frame of each activity are unknown. In the task of activity detection, the purpose is to detect the start and end key frame of each action and also recognize the type of the activity. Many researchers have studied the task of single person action classification in simple scenes. But tasks such as action detection in crowded videos, fast action detection methods, and multi-person activity classification have not been fully discussed. Based on the difficulties in human action recognition, the content of this paper mainly focuses on the following aspects:1) Human action detection in crowded videos:Human action analysis in complex scenes is a challenging task. In this paper, a mask based human action matching method is proposed. It aims to reduce the influence of noise on human action analysis. A complex scene contains various challenging conditions such as occlusions and moving backgrounds. To handle these challenges, a mask is built for template matching. A mask defines a valid region for feature extraction. In this paper, two kinds of masks are built:shape mask and motion mask. In the training stage, we build these masks for each action category. In the evaluation stage, the learnt mask is used for feature filtering and template matching.2) Continuous human action detection in real time:A continuous human action analysis method is proposed in this paper. By continuous, we refer to the video that contains multiple actions in the temporal dimension. By combining the human detection and tracking methods, human actions in a video can be recognized in real time. Compared with the sliding window based methods, our methods runs much faster, and does not have the overlapping problem. Based on the continuous action analysis framework, a translation and rotation invariant probabilistic latent semantic analysis model is proposed. In this model, a category label is added and no EM iterations are required in the evaluation stage. Compared with traditional methods, our method is simpler, and can run in real time.3) Multi-person activity classification:We study the task of multi-person activity analysis. A coupled and observation decomposed hidden Markov model is proposed. In this model, a multi-person activity is analyzed in two levels:the individual level and the interaction level. The individual level reflects the motion character of each individual person such as his/her moving speed and silhouette shape. The interaction level reflects the shared information between two persons such as the distance between them. Compared with the traditional hidden Markov model and its variations, the proposed model in this paper provides more activity details. It has an easier training way towards the changing number of persons, and a more flexible feature selection way.To sum up, we are going to study tasks of human action detection in crowded scenes, continuous human action detection in real time, and multi-person activity classification. A mask based action detection method is proposed for feature filtering in crowded scenes. It aims to improve the action detection accuracy in complex scenes. In order to detect human actions in real time, a continuous human action analysis framework is proposed. It recognizes human actions in real time by combining human detection and tracking techniques. For multi-person activity analysis, a novel model is proposed for activity learning and classification.