Crowd Behavior Understanding And Recognition

Crowd is a common phenomenon in nature and human society, which is referred to as a collection of individuals that are mutual dependence and in?uenced each other.In general, crowd behavior is recognized as relative motions among the individuals.It is not surprising, therefore, that crowd analysis has received attention from technical and social research disciplines. The behavioral analysis of crowded scenes is of great interest in a large number of applications, including visual surveillance, intelligent environments, cytobiology, pharmacology, etc. Although one may think that a straightforward extension of algorithms designed for individual behaviors recognition could be suitable for dealing with the crowded situations, crowd behavior recognition encounters many unsolved challenges due to occlusions among pedestrians, complicated motion patterns in crowded scenarios, unpredicted event evolution, etc. Addressing the aforementioned challenges in crowd behavior for both macroscopic social crowd and microcosmic cell crowd, we proposed a framework for large scale crowd behavior analysis, named Multi-Agent Crowd Modeling, by integrating the achievements in computer vision and machine learning. In particular, the framework consists of the exaction of low level vision feature, Agent partition and attributes recognition, and the high level recognition of crowd behavior. All the ideas, techniques and contributions are summarized as follows:Firstly, we propose a novel spatiotemporal variation ?uid ?eld to model motion patterns of macroscopic social crowd by exploring both appearance and driven factors.Large-scale crowd events are then recognized based on characteristics of the ?uid ?eld.First of all, a spatiotemporal variation matrix is proposed to measure the ?uctuation of video signals in both spatial and temporal domains. After that, eigenvalue analysis is applied on the matrix to exact the principal ?uctuations resulting an abstract ?uid ?eld.Compared with the conventional method, the proposed algorithm does not depend on individual detecting and tracking, thus is more suitable and practical for the analysis of large-scale crowd with median or high density.Secondly, we proposed a novel phase retardation feature to extract e?ective descriptors for microcosmic cells in phase contrast microscopy by analyzing its particular image formation process. Due to the optics principle, computer-aided phase contrast microscopy image analysis is challenged by image qualities and artifacts caused by phase contrast optics. Addressing this issue, we analyze the imaging mechanism of phase contrast microscopy images and construct a dictionary based on di?raction patterns. Afterwards, we formulate and solve a min- 1optimization problem, and a phase contrast microscopy image is represented with a linear combination of bases with top discrimination capabilities. Hence, each pixel is restored into a phase retardation feature vector related to optics. Compared with the intensity in images, phase retardation features are of great relevance to biophysical information and useful for cell category classi?cation.Thirdly, addressing the challenge of granularity for crowd behavior analysis, we proposed a Multi-Agent algorithm to model moving crowd. In order to eliminate the local redundancy and reduce the computational cost, we partitioned the sequence into multiple agents by clustering the neighboring pixels with similar low level features.Therefore, each pixel within an Agent is feature-homogenous, i.e., continuous in temporal domain, similar in spatial domain, and related in logical. Additionally, one Agent has its own visual attributes, and also makes up an entry by interacting with each other.Therefore, a moving crowd is recognized as a set of Agents.Moreover, we proposed an interactive paradigm to recognize the Agent attributes.Addressing the challenge of sample selection for semi-supervised classi?cation, we propose an active sample selection algorithm for annotation by minimizing the prediction error, such that the most informative samples are selected automatically. Then,human interventions are systematically propagated to the unlabeled samples via the a?nity graph. If there exist misclassi?cations during the label propagation, we enrich the theory of graph-based learning such that misclassi?cations are cancelled based on correction propagation. Compared with the previous algorithms for label propagation,the performance of the classi?cation are improved gradually by incorporating the historical results and human intervention. In other words, the graph is becoming smarter by correcting its mistakes, named ”a smarter graph”.Finally, we design an system for crowd behavior recognition by integrating the aforementioned achievements. On the one hand, for the sparse social crowd or cell crowd, we track each Agent by linking the relevant Agents frame-by-frame, and crowd behavior recognition is realized by analyzing the characteristic of the trajectories. On the other hand, we utilized the topic model to map the Agent attributes to the high-level behavior recognition with serious occlusions. Crowd behaviors, e.g., running, splitting,gathering, are recognized e?ectively and e?ciently.In general, we conduct a comprehensive research on both macroscopic social crowd and microcosmic cell crowd in terms of low level feature extraction, motion modeling, and behavior recognition. Although there exist some di?erence in the research objects, they share some comment characteristics in research technique. In the view of observation, the motion characteristics of the social crowd is similar to the cells when it degenerates to spot targets. Therefore, the study on the previous di?erent crowd is complement and promotes each other’s work...