A Visual Localization Technique Based On Street View And Aerial Image Registration

Visual localization is an important research topic in the field of computer vision.With visual information captured by using an image sensor,this task aims to analyze the information for locating the current location on a given map.Based on a study of the existing visual localization approaches,a visual localization scheme based on the registration of street view and aerial image is designed,and two basic algorithms are proposed and investigated.The major contribution of this work includes the following three parts.In order to extract vertical corner lines from street view,a new multi-scale line segment detection algorithm is proposed in the first part of the dissertation.Currently,the popular algorithms detect line segments under one scale parameter only.This leads to unsatisfactory results in term of precision,recall,and other evaluation metrics.For that,a multi-scale image analysis strategy is employed to solve the problem.First,an optimal selection strategy based on image cosine similarity is proposed.By using this strategy,the optimal scale is determined for detecting the straight line segment.Furthermore,a corner recover technique is proposed to improve the integrity of the detected straight line segments at boundary corners.Based on the above improvements,the existing vertical corner line extraction method is further investigated.An interactive extraction strategy is proposed and used to improve the efficiency and stability in complex environments.In order to extract building corners in aerial images,a new algorithm for detecting the feature is proposed in the second part of the dissertation.Since the existing generic corner detection algorithm will get a lot of redundant corners when it is directly used to extract corners of buildings,we incorporate image semantic information into the corner detection process so that building corners and non-building corners can be correctly identified.Specifically,a building area is obtained by using a fully convolutional neural network model trained for regional segmentation of buildings,and then corner detection is conducted on the resulted segmentation map.As a result,redundant corner points can be effectively removed.On the basis of former two parts,a visual localization technique is designed in the third part of the dissertation.First,a circular coding strategy is proposed,with which we can encode the street view image based on the vertical corner lines of the building,and encode the aerial image based on the visible building corners.Then a fuzzy registration strategy based on the above-mentioned circular codes is proposed to achieve a matching between the two types of images.Finally,the visual localization is realized based on the image registration results.This technique employs the coordinate information of the building to achieve the function of locating,and the information has a low degree of repetition.In addition,statistical methods are used to represent the features of maps and location points,and hence the computational complexity is low.Simulation experimental results indicate that the new algorithms are effective.Specifically,the multi-scale line segment detection algorithm outperforms current mainstream algorithms on the main detection index,and has stronger anti-noise performance;Compared with the generic corner detection algorithm,the proposed building corner detection algorithm can solve the problem more effectively.These characteristics and advantages ensure the feasibility and stability of the final visual localization technique.