Research Of Place Recognition Approaches Based On Image Match Using Convolutional Features

With the development of mobile robots,unmanned driving and other new fields,vision-based navigation and positioning technology has become an important research direction in the field of computer vision.Visual place recognition studies how to use image information to determine which map scene a place belongs to.It plays an important role in vision-based topology location and simultaneous localization and mapping(SLAM).However,due to viewpoint difference and continuous changes of the external environment,visual place recognition becomes very challenging.Considering the robustness in the complex environment,traditional recognition approach based on bag of word model is sensitive to the texture information,which makes it difficult to meet the requirements of the current navigation and positioning system.In recent years,the development and application of computer vision,especially convolutional neural network,has greatly promoted the development of visual place recognition approaches.Scholars have begun to study how to use convolutional features and semantic information of images for high-level place recognition.This paper studies place recognition approaches based on image match using convolutional features.The main contents and research results are as follows:(1)A method of generating visual words using convolutional features is proposed(CNNW),as well as a geometric consistency verification strategy based on hypercolumn vectors,to improve the sensitivity of the original convolutional features to the change of viewpoint.CNNW combines approaches based on convolutional features with bag of word model,and generates visual words by separately clustering of different channels of convolutional features.Image similarity is computed according to term frequency-inverse document frequency.Based on CNNW,the method of constructing word pairs according to adjacency position is further designed.In the geometric consistency verification stage,the hypercolumn vectors of the lower network layer are used to extract and match the salient regions of the image.The geometric consistency between images is calculated according to the proportion of the inliers and outliers.Experiments have shown that compared with the original convolutional features,CNNW not only retains the robustness of convolutional features to the change of appearance,but also enhances its ability to cope with the change of viewpoints.Besides,the construction of word pairs can improve the discrimination degree of different places.Geometric consistency verification can effectively eliminate the false recognition points and improve the accuracy of place recognition.(2)A place recognition approach based on convolutional feature map landmark is proposed to improve the place recognition ability in case of severe environmental changes.Based on the research of geometric consistency verification using convolutional features,the convolutional feature maps are directly used to calculate the energy response of the image.The landmark extraction and descriptor generation are based on the maximum point of the energy map,as well as the projected position of the high layer feature maps.In the stage of landmark matching,a new matching approach named MRF-Bayes matching is designed.This approach exploits probabilistic graphical model and Bayes filter to build matching model,in which the geometric information of landmarks is integrated.The strategy of separate calculation of seed nodes and remaining nodes ensures the efficiency of model solution.Experiments have shown that compared to EdgeBoxes and super pixel segmentation,the proposed approach improves the efficiency of landmark extraction and ensures the repeatability of landmarks.When the images have a strong appearance changes,the matched landmarks still have a high geometric consistency,which ensures the accuracy of place recognition.(3)A light weight convolutional neural network for visual place recognition named Octave Place Recognition Net(OPRNet)is designed and trained to improve the generation speed of convolutional features.OPRNet is based on the theory of octave convolution and uses simplified octave convolution unit for network construction.Multi-scale feature maps replace the traditional feature maps,and the scene classification dataset replaces the object recognition dataset during the training stage.Experiments have shown that OPRNet reduces the weight of the network and improves the generation speed of convolutional features,while ensuring the accuracy of place recognition.(4)A place sequence recognition approach based on fusion convolutional features is proposed to improve traditional place sequence matching.Multi-scale fusion features are used to construct similarity matrix.In order to fuse sequential information,bayesian model is constructed to transform the similarity matrix into probability matrix.Aiming at the problem of overconfidence in bayesian filter,priori probability penalty and bidirectional filter are proposed.Experiments have shown that this approach effectively solves the limitations of traditional sequence matching approaches,and improves the accuracy of place recognition based on sequence matching.To sum up,CNNW has better place recognition performance than bag of word model and original convolutional features.The proposed approach based on convolutional feature map landmark can directly extract landmarks based on convolutional features,and integrate the geometric consistency of landmarks into similarity calculation to ensure the recognition performance when strong environmental changes exist.OPRNet further improves the problems of too much weight and slow forward propagation of feature generation network.Place sequence recognition approach based on fusion convolutional features improves traditional sequence searching pattern.The theory and algorithm in this paper have been verified on the real datasets and compared with the classical place recognition approaches,which proves the superiority and practical application value of the proposed approaches.Thus,the paper can provide some basic theory and technical support for autonomous navigation of robot,unmanned driving,and other fields.