Video Event Retrieval Based On Local Space-time Feature

Local space-time features have become a popular video representation for eventrecognition. Relatively speaking, accurate localization and background subtractionare not required and local representations are less sensitive to noise and partialocclusion. Harris3D detector can solve the problem of multiple scales, moreover,the local space-time features can be adapted to size and velocity of the pattern of theevent. In additional, jet responses of the space-time interest points capturecharacteristic shape and motion in video and provide relatively independentrepresentation of events. We explore and study the technique of video event retrievalbased on local space-time features in this paper, and the major contents are listed asfollow:1. A method of event recognition based on self-organization map analysis hasbeen proposed.The number of frames of each video clip is different and the quantity of thespace-time interest points of each frame is also diverse, so in order to unify therepresentation of video sequence, we adopt the idea of Self-Organizing MapAnalysis. For every event class local spatio-temporal features used for training werefirst trained to construct a neural map which was a2D plane with a lattice on it, anda video is then represented as SOM density map.2. A method of event recognition based on bag-of-feature has been proposed.In addition to self-organization feature map, we also adopt the idea ofbag-of-feature. For every event class local spatio-temporal features used for trainingare first quantized into visual words and a video is then represented as the frequencyhistogram over the visual words. 3. A method of event recognition based on the combination of audio and videofeatures has been proposed.Auditory information can enhance people’s perception, so audio is anotherimportant aspect of the video; it can provide useful features for the event recognition.Therefore, we also use the audio feature (MFCC) which combines with localspace-time features for event recognition. These two features use self-organizationfeature map training respectively to get their own neural map. Finally we combineSOM density map of local space-time feature with SOM density map of MFCC toform the feature vector of the video.To evaluate effectiveness of these methods, this paper uses the publicHollywood dataset, in this dataset the shot sequences has collected from32differentHollywood movies and it includes8event classes. The presented result justify theproposed method explicitly improve the average accuracy and average precisioncompared to other relative approaches.