Geo-tagging For Large-scale Multimedia Data And Its Applications

With the rapid growth of techniques including computer science, electronic communication and multimedia technique, people can obtain information ans share them with other users on the Internet coviniently. The explosive growth of information on the Internet, brings abundant information resources for people. Image and video make up most of the internet traffic, thus the organizing and management of the large amount of multimedia data is one of the key problems that have draw lots of attention from both industry and academia. Geo-tagging, which aims to add geographical identification metadata to the multimedia data, can help users find a wide variety of location specific information. It is also benificial to the storage and visualization of these data. However, it becomes increasingly challenging to manage such an overwhelming amount of multimedia data. Not only the approximate position, but also other geographical information, including camera position, camera viewing orientation, the scene location and more specific geometric structure information, is needed for further application such as virtual navigation. In this paper, we propose a novel content-based localization approach which aligns the2D image to3D scene models to calculate the geographic information.In this paper we focus on technique about content-based image geo-tagging, including the estimation of comprehensive geographic parameters, the optimization of localization results and the applications in image inpainting with internet photos. The contribution of this thesis can be summarized as follows.Firstly, we propose a novel visual-based localization method that estimates the comprehensive geographic parameters of the given image.3D scene models are obtained by reconstruction from image clusters. For a given query image, similar images are retrived and then used to vote for related3D scene model. Finally the2D image is aligned to the3D scene model for localization. The estimated geographical parameters include the camera location, viewing direction and scene location. This comprehensive information can be used for mobile applications such as virtual navigation to help user get a better understanding of his surrounding.Secondly, we propose an optimization method to enhance the accuracy of geo-tagging. We propose a scheme to efficiently generate visual codebooks with strong discriminative power of different locations. Using the geo-tags of the database image as a prior knowledge, we calculate the geographic distribution of each visual word to measure their discriminative power. We get better location recognition performance with the proposed visual word weighting scheme. Furthermore, we propose to analyze the query image for more specific structure of the scene, leading to more precise geo-tagging of the image.Thirdly, we explore the application of geo-tagging in image processing. We present an image completion method that replaces a specified region of photographs using other reference photographs from Internet. We search candidate images that capture the same scene or building from the Internet using image geo-tagging. Then we establish geometric relationships between candidate images and the query image. The geometric relationships are represented by homography transformations estimated using viewpoint invariant local feature matches. Given these transformations, we can project the structure information from the candidate images to the target image. The extracted structure information includes line structures and region segmentation information, which are very helpful for image completion. Finally, we use such structure information for image inpainting to get fine-grained image completion results.In a nutshell, in this thesis, we explore and discuss techniques about geo-tagging for large amount multimedia data on the Internet from novel and distinctive perspectives and propose several applications based on geo-tagging. Compreshensive experiments demonstrate the effectiveness and efficiency of proposed algorithms.