Research And Application On Spatio-temporal Analysis Of Multi-view Videos

Spatio-temporal analysis is a process of extracting features by analyzing spatio-temporal data, in other words, it is a process of extracting important parts of data from overall data and apply them to other research fields. Spatio-temporal analysis based on multi-view videos can extract spatio-temporal features from videos, which is helpful for us to analyze the action of object, determine semantics and understand scene, so it is one of the most important researching issue in computer vision field. Spatio-temporal analysis based on multi-view videos which needs to finish the process of object detection, matching, position prediction from videos is a big challenge task. This paper focus on Multi-egocentric or multi-view egocentric videos, study of spatio-temporal analysis methods based on features of Multi-egocentric videos and an application of group discovery based on Multi-egocentric videos using spatio-temporal analysis methods.In this paper, firstly for input, we use image sequences from multiple egocentric videos which are made in the same period of time. Secondly, we use spatio-temporal analysis methods for object detection, object matching, and estimation of object’s position and orientation. Finally, we finish the task of group discovery for output. The main work of this paper is as follows:Firstly, we study spatio-temporal methods for multi-view videos. In order to overcome the difficulties of Multi-egocentric videos including violent background variation, distinct difference of scales of objects and the real-time changes of view, we build multi-object detection model based on boosting method to detect objects roughly from multiple videos and propose a local similarity based region optimization algorithm, also using spatio-temporal information, to refine the contour of each object. The results of experiment on Party Scene dataset prove that our algorithm solves the problem of detection inaccuracy caused by egocentric videos, and is more robust than Zhu’s saliency optimization algorithm based on background measure.Secondly, we study algorithms of object matching, position and orientation estimation based on multi-view videos, using one object’s multiple view images as input, train a SVM classifier by HOG feature with spatial pyramid method to match multiple objects from multiple videos. Based on features of egocentric videos, we use Ego-motion information to classify motion mode of object, and propose an algorithm of estimating orientation and distance of object’s movement based on optical flow’s orientation and volume. The results of experiment on Party Scene dataset prove that our algorithms’ performance is as good as the same type algorithms’. But considering egocentric video’s features, such as distinct difference of scales of objects, real-time changes of view, our algorithms meet more difficulties which match our expectation.Finally, we study methods of group discovery from Multi-egocentric videos, and contribute a novel method to model individuals’ attention at any moment with spatio-temporal information, which successfully measures individual pairs’ interrelation. We use adaptive clustering, where the number of groups is determined by minimizing the proposed cost function inspired by the biological constraints of the plausible zone to calculate results of group discovery. The results of experiment on Party Scene dataset prove that our algorithm’s performance is better than IRPM, GCFF and GTCG for Multi-egocentric videos.