Research On Hierarchical Image Retrieval And Its Applications Based On Local And Global Features

Content Based Image Retrieval(CBIR)is one of the supporting technologies for fast determination of image relationship in Near-Real-Time Photogrammetry,Visual Localization and Loop Detection in Visual Simultaneous Localization and Mapping(VSLAM).CBIR firstly acquires the image features of the query image and the database image,generates the feature descriptions representing the original images from these features,and then indexes the query based on the feature descriptions.However,the above applications require accuracy and real-time,but image retrieval methods based only on traditional features cannot meet such requirements in terms of efficiency and precision.Meanwhile,CNN have performed well in a number of computer vision applications in recent years.In this paper,to meet the efficiency and precision requirements of visual localization and Near Real-time photogrammetry,a hierarchical image retrieval technique combining local features extracted by SIFT(Scale Invariant Feature Transform)or Super Point with global feature vectors acquired by Res Net101 or VGG16+ Net VLAD is studied,so as to improve the efficiency and precision of the above applications.The core idea of "hierarchy" is: firstly,several candidate images are selected based on cosine distance between efficient and fixedlength global feature vectors,and then candidate images are reranked based on local feature.It not only avoids the time consuming of using a single local feature retrieval method in a large range,but also further optimizes the retrieval results in a small range by using the local feature method.The main work of this paper is as follows:?.Design and verify the accuracy and efficiency of three hierarchical image retrieval schemes.In the first scheme,Top-N images are quickly selected based on cosine distance between global feature vectors extracted by Resnet101,and then candidate images are reranked based on SIFT matching.In scheme 2,Top-N images are quickly selected based on cosine distance between global feature vectors extracted and clustered by VGG16 and Net VLAD,and then candidate images are reranked based on Super Point matching.Scheme 3 firstly uses YOLOv3 to remove interferences unrelated to the main content of the image and fills the corresponding areas white,the next steps are the same as Scheme 2.?.Based on the optimal scheme,a hierarchical retrieval scheme suitable for visual positioning is designed firstly,and the positioning accuracy and efficiency of the scheme are tested in indoor.Secondly,a dynamic hierarchical image retrieval scheme suitable for Near Real-time photogrammetry is designed.On the basis of the original optimal scheme,the dynamic expansion of database and the dynamic image relationship construction are considered.In the experiments,this paper compares and tests the efficiency and effect of SIFT and Super Point.Then it is verified that removal of interferences by YOLOv3 can improve the retrieval accuracy and have negligible impact on local feature extraction in hierarchical image retrieval scheme.And three hierarchical image retrieval schemes are tested on benchmark dataset and the real scene dataset.Finally,the optimal scheme is applied to the visual localization Near Real-time photogrammetry.