Research On Detection And Analysis Method For Video Semantic Events

With the rapid development of the applications of network, enormous multimedia files are emerging every day. Video data is an integration of text, sound, image and other files. Not only being hierarchical, structural and complex, video data is also rich in semantic information. Therefore, extensive attentions have been drawn to how to process video data quickly, extract the video characteristics accurately, and analyze and understand the semantic content deeply. Semantic event detection and analysis could find video information quickly and accurately for users in the vast ocean of video data. It can be applied to the field of video on demand, intelligent monitoring and video mining as well. However, there are still many limitations, such as low recognition rate for multiple moving objects with different characteristics, low accuracy in semantic event detection, difficulties in detecting semantic event correlations, the lack of consistent standards of event semantic description, and so on.This dissertation focuses on four key techniques, the description and classification method of multiple moving objects, detection and analysis of complex semantic-based events, semantic event correlation mining, and the description and understanding events. This work then discusses their research status and potential problems. The main contributions are listed below.(1) The recognition methods of multi-moving objects based on adaptive combined moment invariants are proposed. In this method, the adaptive combination of moment invariants metric technique is proposed to describe different characteristics of different objects with selecting dynamically invariant moments. Then, Similar Frequency-Inverse Singular Frequency (SF-ISF) is defined to calculate the moment invariant weight value of each object. Support Vector Regression (SVR) model of multi-class classifier is then built to classify a variety of moving objects in the scene, with weight value and moment invariants value as an input parameter. Experimental results show that the method of selection moment invariant metric, with weight, is reasonable. The performance of multi-class classifier SVR is verified by experiment that it improve the recognition rate of multi-moving objects effectively.(2) The detection and analysis methods of complex event based on matching integration between trajectory and multi-label hypergraphs are proposed. Firstly, sub-events are detected by definition, prune, normalized cut and similarity calculation of trajectories. Secondly, trajectory and multi-label hypergraphs are constructed for classifying and recognizing the complex events. By matching the trajectory hypergraph and multi-label hypergraph, mapping relationship between trajectory and multiple semantic labels is built to extract the complex semantic events. The recognition of low-level features to high-level semantic is made possible for video events. Experimental results and related analysis show that this method can effectively improve the average precision and average recall rate of video complex event detection.(3) The mining algorithm of semantics events based on temporal association rules is proposed. According to the temporal association between the motion laws of multiple moving objects in an event, a frequent pattern tree structure is designed for describing events, storing events semantic labels. Then, the weighted frequent pattern growth method is designed. It mines the frequency of events as well as the strong association rules between events with temporal characteristics. These methods overcome the shortcomings of low accuracy in semantic event detection. It can also filter the non-frequent item set and find the events with temporal association. The experimental results show more accurate numbers of strong association rules obtained by this method, the algorithm operating efficiency is improved considerably.(4) The methods for description and understanding of semantic events based on the case grammar framework network structure are proposed. Case grammar theory is used for natural language understanding. A design of case grammar framework network structure (CSFN) which incorporates the case grammar and semantic event structure features to describe the relationship between the sub-events. There are seven types of relationship defined as Inheritance, Subframe, Temporal, StartState_EndState, Causative, Using, and Ref_Asso, which describe the relationship among objects, events, and states. In Ref_Asso, the temporal and spatial association is analyzed, for describing the relationship of multi-threaded event. Experimental results show that with the semantic description of the multi-threaded event by the case grammar network structure, it is better for users to understand the semantics and the effectiveness of semantic event detection is improved. In this dissertation, we have researched on the aspects of the description and classification method of multiple moving objects, the detection and analysis method of complex event semantic-based, the mining algorithm of semantic event temporal association rules, and the methods of description and understanding for semantic event. These methods will provide the foundation and references to the further research on the semantic events recognition and analysis.