Structured Cognitive Computing Based Crowd Behavior Analysis

With the rapid growth of population,more diverse activities and socialization process,crowded scenes are becoming more common which result in great demand of modeling and understanding the crowd behavior.Compared to the previous works on video content analysis,the factors that large population and complex scenes make crowd behavior analysis face great challenges.Meanwhile,crowd behavior contains important clues for a lot of interdisciplinary field problems,understanding the formation mechanism of crowd behavior has become one of the important research subjects of sociology and natural science.Crowd behavior analysis of the research can provide the support for many key engineering applications and the corresponding solutions,such as intelligent video surveillance,abnormal crowd monitoring,public facilities planning,etc.This makes high-level semantic understanding and analysis of the crowd behavior more and more urgent.For the analysis of crowd behavior in videos,referred to as “crowd behavior analysis”,the main goal is understanding and analysis of semantic content in crowded scene based on ordinary monitoring videos.On the basis of analysis about the present situation of researches,we found that the development of the existing methods mainly restricted by two challenges,mainly lack of crowd cognitive mechanism and absence of structural semantics.This dissertation starts with structured cognitive information in the role of crowd behavior analysis,studies effective computing framework and algorithm for crowd behavior in the stages of representation,collaboration and mining structured cognitive information as the main line: based on the structural interaction attributes,we expect to obtain representation model for describing the interaction of crowd behavior.Using structured semantic information for group modeling,we investigate group collaboration model with consistency of co-occurrence and formation structures and fusion of multiple attribute.Towards high-level semantic knowledge of crowd behavior,we mine the crowd mood and crowd saliency mechanism.We study the applications in the field of intelligent video content analysis of crowd behavior,trying to mine the dynamic crowd patterns and behavior in video data of real scene.Specifically,the main contents and contributions of this dissertation can be summarized as the following:Firstly,for the lack of the cognitive mechanism of crowd behavior,namely “the semantic cognitive mechanism is needed to fill the semantic gap between the characteristics of the underlying movements and the high-level group behavior” problem,we propose a structural interaction attribute based cognitive representation for crowd behavior,which captures crowd behavior interaction to enhance determinativeness and richness of existing representation.Existing representation models of crowd behavior lack of deeper modeling of social interaction,which need to resort to build the semantic representations from the underlying motion description to the middle-level interaction based on attributes or concept properties.By reference to social cognitive mechanism of crowd behavior,this paper systematically puts forward the structural interaction properties of organization and representation method,uses quantitative attributes as group representation,and proposed online fusion strategy for structured attributes.Experimental evaluations are conducted on UMN?UCSD?UCF-Web datasets for crowd abnormal detection with other comparison methods.The results demonstrate the effectiveness of the proposed model.Secondly,for the absence of structured semantic information in crowd behavior,namely “How to make use of the structured semantics of the groups and associated features and multiple attributes” problem,we expand the research to improve the semantic representation of crowd behavior with collaborative modeling.We propose structure consistent graph mining method for group detection,which includes co-occurrence structure consistent Bag of Trajectory Graph Model to recognize crowd event,and formation structure consistent dense subgraph seeking model.The experiments on UMN and PETS dataset shows the proposed method can effectively improve performance of event identification and group detection.Furthermore,comprehensive description of the crowd contains a variety of properties including homogeneous,heterogeneous,and topological attributes.This paper investigates how to fuse multiple attributes and proposes deep attribute-embedding graph ranking method for crowd video retrieval.The proposed method integrates multiple attributes into graph ranking framework,simultaneously optimizes the ranking score,attributes weights and deep transform metric.The experiments for crowd video retrieval on CUHK-Crowd dataset show the proposed method obtains the optimal performance.Finally,this paper integrates basis of structured cognitive representation,proposes high-level semantic knowledge mining for crowd behavior including crowd mood modeling and crowd saliency mechanism modeling.For crowd mood,This paper investigates features in structured trajectories and mapping relation with emotional space,and puts forward modeling method based on structured trajectory learning for crowd mood.To extract features of coherent trajectories by structured trajectory learning,we further utilize weighted regression learning to mapping the features into emotional space to build crowd mood curves.The experimental results show that the proposed representation is effectively capable of crowd mood classification and matching tasks.In addition,from the point of modeling crowd saliency,we investigate the crowd saliency mechanism and propose two-stage cascaded deep networks for crowd saliency modeling.The results show our method jointly considers the crowd and saliency perception,which is effective compared with existing methods.Through the above studies,this dissertation deeply explores the crowd behavior representation and computing model towards video content understanding,providing feasible and effective solutions towards the key technical issues of crowd behavior analysis.The experimental results show that: the structure cognitive factors play important roles in crowd behavior representation and applications.Incorporating structured interaction attributes can help to obtain richer and more understandable features to enhance the objectlevel description,and improve the accuracy for abnormal detection.Structured semantic for specific group behavior contains co-occurrence and formation structure consistence.Considering consistency and multiple attributes collaboratively,the problem can be significantly improved in group pattern analysis.Combining with the cognitive mechanism of crowd behavior,emotion and saliency mechanism of high-level semantics can be further interpreted and modeled.Meanwhile,the solutions can be effectively applied to the practical tasks such as crowd event recognition,crowd mood classification and crowd saliency prediction,etc.