Leveraging georeferenced meta-data for the management of large video collections

The rapid adoption and deployment of ubiquitous video sensors has led to the collection of voluminous amounts of data and hence there is an increasing need for techniques that manage these collections in a variety of applications, including surveillance and monitoring systems, web-based video search engines, among others. However, the indexing and search of large video databases remains a very challenging task. Current techniques that extract features purely based on the visual signals of a video are struggling to achieve good results, particularly on the large scale. By considering video related meta-data information more relevant and precisely delimited search results can be obtained.;Latest technological trends have enabled the cost- and energy- efficient deployment of video cameras together with other sensors (e.g., GPS and compass units). The sensor data acquired automatically in conjunction of videos provides important cues about the video content. In this dissertation we propose to utilize the location and direction meta-data (i.e., georeferenced metadata) from the GPS and compass sensors to describe the coverage area of mobile video scenes. Specifically, we put forward a viewable scene model which describes the video scenes as spatial objects such that large video collections can be organized, indexed and searched effectively using this model. Our work focuses on the following key issues in leveraging geo- referenced meta-data for effective search of large video collections: (1) Acquisition of meta-data from sensors. We develop a prototype system to acquire georeferenced meta-data from GPS and compass sensors. The proposed system addresses several challenges in acquiring sensor meta-data including, compatibility among various meta-data streams, synchronization with video frames, etc. (2) Management and search of georeferenced meta-data. We propose a viewable scene model for the automatic annotation of video clips and present the algorithms for searching videos based on this scene model. We also propose a novel vector based indexing for the efficient search of large video collections. (3) Presentation of search results. We investigate and present three ranking algorithms that use spatial and temporal video properties to effectively rank search results. Finally, we introduce a prototype of a web-based georeferenced video search engine (GRVS) that utilizes the proposed viewable scene model for efficient video search.