Research On Interaction Analysis Based On Complex Causality Relationships

Interaction is an important component of non-verbal communications in realistic video.Internal understanding of interaction consists of two parts: individual motions of entities,and the interrelationship between individuals' movements under specific spatio-temporal constraints.As complex activity is composed of interactive behaviors,interaction analysis bridges the simple action recognition and complex activity analysis,which is the key cue in activity recognition.Therefore,it has widely applications in video surveillance,video summarization and video indexing.Though the existing models could achieve promising performance for simple interaction analysis and recognition,they are limited by the complexity of interactions.The complexity is reflected in the complex structure representation and the complex spatio-temporal context:(1)the interaction is performed by two or more individuals;(2)intentional interactions could co-occur simultaneously;(3)the interactive relations between individuals could be dynamic over time.In this thesis,we firstly summarize the methods of modelling structure information utilized in activity analysis.Combined with important characteristics of pairwise interactions in multi-agents group activity,we focus on the following difficulties in group analysis: Firstly,it is difficult to realize group detection and group activity recognition,simultaneously.Secondly,when multiple group activities co-occur in the scene,there are few methods to tackle the activity recognition,respectively.Thirdly,as interactive context is vital structural cue for multi-agents group analysis,it is critical to capture the context comprehensively and accurately for analyzing the temporal and semantic relations amongst some actions,which contributes to extract key frames in video summarization.In light of aforementioned difficulties,the main work is as follows:(1)As for the difficulty of simultaneous accomplishing group detection and activity recognition,we mine the casual interactive context by the Granger causality.Based on the context,with the help of independent analysis between group interaction patterns and individual behavior patterns,group memberships are identified to realize the group detection;with the hierarchical extension of topic model,the interactive context is integrated into the framework to model group activity.(2)For the issue of respectively recognizing co-occurring group activities,we utilize the interactive context to restrict the motions into a specific spatio-temporal group volume.With the spatio-temporal constraint,the specific group activity can be modelled by the proposed method.(3)To solve the information redundancy and the semantic incoherence induced by the clustering methods without considering the temporal structure relation,a framework is utilized to integrate the interactive context and other constraints used in video summarization.To retain the original video information to the maximum extent,the framework can realize the redundant information of summary minimized,and the length of summary can be adjusted dynamically according to the video.