Cross-scene Crowd Behavior Understanding Based On Computer Vision

With the deepening of global urbanization,the urban population becomes increasingly dense,and large-scale activities and events occur more and more frequently.Crowd behavior understanding has received extensive attention from sociology,psychology,biological science and other disciplines due to its demand in intelligent video monitoring,public space management,urban public security and other fields.The content of this article is study and understand the crowd behavior from the research field of computer vision and machine learning based on large scale video data,including crowd segmentation,crowding behavior attributes estimation,crowd counting estimation,the crowd behavior type recognition,etc.The key points which affect crowd behavior understanding and analysis include large data sets supporting,specific feature design for crowd behavior and specific statistical model design and development focus on specific crowd characteristics.This paper has carried out relevant research around the above key technology links and achieved some innovations.Firstly,we collate and discloses the large-scale cross-scene crowd behavior understanding dataset WorldExpo'10,which is the biggest real crowd scenario video dataset currently exposed.The data were collected from the real monitoring video data of Shanghai WorldExpo 2010 and we sorted out the data systematically and desensitized these data.WorldExpo'10 dataset contains two sub-datasets,which are cross-scene crowd behavior attribute estimation dataset and cross-scene crowd counting dataset respectively.The dataset provides specific label tools with detailed annotation to handle the task of understanding of different crowd behaviors.Four different crowd attribute prediction tasks are proposed in the cross-scene crowd behavior attribute estimation dataset,including crowd segmentation,crowd density estimation,crowd collectiveness estimation and crowd cohesiveness estimation.The cross-scene crowd counting dataset is built for task of the crowd counting.Since its publication,WorldExpo'10 dataset has received wide attention,ranging from over 1000 downloads and 350 academic citations,and it is widely used for validation and evaluation of algorithms such as crowd behavior understanding.In order to solve the challenge of crowd behavior understanding proposed by WorldExpo'10 effectively,this paper proposes a crowd behavior attribute estimation algorithm based on multi-scale spatio-temporal similarity.This paper designs two crowd features to describe the characteristics of crowd global scene and crowd local behavior,which are global crowd feature and local crowd feature.The global crowd feature is used to represent the crowd scene information and by which we can match the similar scene.Local crowd characteristics represent the specific local crowd behavior characteristics and they are used to match the scene with most similar local features.In addition,the relevance feedback algorithm is used at the same time to adjust the parameters of local characteristics of the crowd,which makes it more suitable to describe the characteristics of the crowd.Four different crowd attributes can be estimated simultaneously by data-driven tag transfer method based on the multi-scale similarity of two crowds.Experimental results show that the non-parameter data-driven crowd attribute prediction method is effective in crowd segmentation,crowd density estimation,crowd collectiveness estimation and crowd cohesiveness estimation,and its performance is better than other algorithms.A relative comparison algorithm is specially designed for each specific attribute estimation,while the proposed data-driven crowd attribute estimation algorithm can complete four attribute estimation tasks simultaneously under the same algorithm framework.To solve the task of crowd counting under cross-scenario,this paper proposes a multi-task crowd convolutional neural network model to realize the description of crowd,which has stronger expression ability and discrimination ability than other commonly human-designed crowd description features.The proposed cross-scene crowd counting algorithm can realize the crowd counting task under any target scene without extra annotation.In addition,we propose a target switchable training method,which is able to transform the crowd counting task into two specific tasks which are crowd density distribution estimation and crowd global counting.The expression ability of the crowd convolutional neural network is enhanced through the alternating training of these two tasks.In order to make the model more suitable for the new test scenario,a data-driven model fine-tuning algorithm is proposed.A more accurate cross-scene crowd count can be achieved by fine-tuning the convolutional neural network model of the crowd through retrieving data similar to the test scenario.Experimental results verifies that the crowding convolutional neural network algorithm proposed in this paper can achieve the goal of cross-scene crowding,and is superior to other published algorithms in both multi-scene and single-scene experimental environments.At the end of the paper,we studied the structural grouping of behavior recognition and crowd analysis.The main idea is to transform the crowd to multiple group hierarchy structure.This paper proposes two kinds of crowd behavior recognition algorithms,both of which focus on judging and identifying crowd behavior by studying the interaction between different individuals and groups in the crowd.The first algorithm designs a set of crowd causality characteristics to describe the interaction between individuals,groups and crowds through the response value of the causal filter based on the track information of individual motion.By encoding the causal features of crowds with localizing the constraint,we can ensure that consistent dimensional feature expressions can be obtained even for all crowd scenarios.However,the first grouping method has some disadvantages.It only considers the location information but ignored the movement information between individuals in the grouping.It also fails to adapt to the crowd behavior scenarios with more variable number of people and number of groups.Therefore,this paper proposes a second structured crowding behavior recognition algorithm,which abstracts the crowding behavior into a multi-layer graph model with hidden variables.The implicit latent conditional random field model can be used to identify the crowd behavior type,the behavior of groups and individuals and the grouping relationship within the crowd.A inference method is designed for this structured model,especially a heuristic search algorithm based on sampling is introduced to realize the inference of more reasonable grouping relationship of individuals in the crowd.And finally the implicit support vector machine method is used in this model.Experimental results show that the introducing of the concept of group and the analyzing the interaction between groups can effectively improve the accuracy of the crowd behavior recognition algorithm.In addition,the accuracy achieves a significant improvement compared with other published algorithms.At the same time,because the second algorithm makes a more effective inference on the grouping relationship in the crowd,comparing with the first algorithm and other comparison algorithms,the accuracy of crowd behavior recognition is further improved.