Research On UAV Image Geo-localization Technology Based On Semantic Matching

UAV(Unmanned Aerial Vehicle)image has the characteristics of low acquisition cost,high resolution,and time-intensive.It is widely used in social-economic and military fields.Rapid and accurate acquisition of the location of UAV images is the basis and key of various UAV applications.However,the existing methods based on collinear equations have poor location accuracy,and the methods based on 3D reconstruction take a long time,which can not meet the needs of rescue,disaster relief,emergency mapping,and other scenes.In order to overcome these difficulties,this thesis studies the semantic feature matching based UAV image geo-localization in emergencies,by the two semantic features of roads and buildings.The main contributions are:1.For the requirements of UAV image geo-localization,two road extraction methods based on weak/semi supervision are proposed,which reduce the excessive dependence of the model on the precisely labeled data and improve the accuracy and practicability of road extraction.Traditional road extraction methods usually need a large number of precisely labeled data.In order to reduce the dependence on labeling,this thesis proposes a weakly supervised road extraction method(MD-Res Unet)only using the road centerline as the label,achieving the road extraction performance similar to the existing fully supervised methods.Secondly,a road extraction method based on semisupervised(SW-GAN)is proposed in the case of a small number of precisely labeled data.The method integrates the semi-supervised road extraction problem into the generative adversarial network to further improve the road extraction performance of UAV images.2.For the requirements of UAV image geo-localization,an edge-enhanced building extraction method for UAV image is proposed,which lays a foundation for the semantic feature matching based UAV image geo-localization.In order to extract the building semantic features with clear edges and boundaries,this thesis proposes a semantic feature extraction method Es Att-GAN.Firstly,This method uses the attention mechanism to replace the bridge in Unet to improve the feature representation ability of the low-level encoder,and then uses the GAN framework to strengthen the edge feature extraction effect.Therefore,the extracted building features have clear edges and boundaries,which is more suitable for the needs of UAV image geo-localization.3.To achieve UAV image geo-localization in emergencies,a semantic feature matching based UAV image geo-localization method on the premise of semantic features of buildings and roads is proposed to effectively improve the accuracy of UAV image location.Firstly,a baseline-based semantic feature description model is proposed.The model regards the building center and road intersection as semantic feature points and constructs a point-to-surface semantic feature description model based on the semantic feature relationship between the feature points and their surroundings.This method not only has rotation invariance and scale invariance,but also can describe the geometric topological relations between semantic features effectively.Secondly,on the basis of the constructed semantic feature description model,an optimal global registration based semantic feature matching algorithm is proposed,realizing the UAV image geolocalization effectively and meeting the needs in emergency scenes.4.For the scenarios lack of semantic features or with unclear semantic features,this thesis proposes a UAV image geo-localization optimization method(MSLK)and a sequence UAV image geo-localization method(VMSLK)which integrate relative and absolute localization into UAV image geo-localization.Firstly,a multi-stage Lucas Kanade(MSLK)algorithm is proposed using the global information of UAV images.This method uses the deep end-to-end network to solve the problem of inconsistent features of heterogeneous images,and the multi-stage Lucas Kanade method optimizes the location of a single UAV image,improving the geo-localization accuracy effectively.On this basis,the VO algorithm and the geo-localization results of the previous frame are used to realize the initial geo-localization of the UAV image,and the MSLK algorithm is used to realize the absolute UAV image geo-localization in the current frame.The combination of relative and absolute geo-localization technology can better use the dependence of sequence UAV images in time and space dimensions,which can reduce the time drift phenomenon in the geo-localization process and improve the geolocalization accuracy of sequence UAV images.