Landmarks Based On Convolutional Neural Network Associated With Semantic And Spatiotemporal Information Based Visual Place Recognition

As one of the key technologies of visual simultaneous localization and mapping,visual place recognition technology has become a research hotspot in recent years.The current method of Conv Net landmarks based on single-frame image(Conv Net landmark)has attracted widespread attention because of its good robustness to changes in viewpoint and appearance.However,when faced with a complex environment,there is a gap between the repeatability of Conv Net landmarks detected based on a single frame image and actual application requirements.The image sequence can provide multi-period and multi-angle information of a target,which helps to filter out more repeatable Conv Net landmarks.In order to improve the performance of visual place recognition in complex environments,this paper focuses on the combination of Conv Net landmarks and image sequences,and has achieved the following innovative results:In order to cope with more complex environmental changes,a visual place description algorithm based on landmarks associated with spatiotemporal and semantic information is proposed.Conv Net landmarks that are stable to changes in viewpoint and appearance are combined with image sequences with multi-view and multi-period information to construct a visual place descriptor with higher discriminative ability.Through testing on eight public datasets,it is verified that the visual place description algorithm based on landmarks associated with spatiotemporal and semantic information proposed in this paper has a significant performance improvement when dealing with a variety of environment and viewpoint changes.In order to improve the computational efficiency and ensure the real-time performance of the unmanned vehicle system,a visual place matching algorithm based on graph convolutional neural network is proposed.The image sequence features composed of disorderly arranged landmark features are generated by graph convolutional neural network.The compact vector is used in the coarse matching stage to find N nearest neighbors.At the same time,in order to further improve the accuracy of visual place recognition in the matching stage,a two-stage fusion visual place matching algorithm is proposed.The similarity of the visual place matching stage based on graph convolutional neural network and the similarity of the visual place matching stage based on landmark spatial information and geometric information are weighted and summed.Through testing on public datasets,it is verified that the visual place matching algorithm based on graph convolutional neural network has a significant improvement in time efficiency and the two-stage fusion visual location matching algorithm has improved accuracy.In order to verify the effectiveness of the visual place recognition algorithm proposed in this paper in real scenes,a lightweight unmanned vehicle system based on pure visual navigation is designed and implemented.The unmanned vehicle system consists of three simple modules with independent functions and clear logic.This low-coupling design with strictly differentiated functions reduces the complexity of the system on the one hand and facilitates secondary development.On the other hand,it can simplify the hardware design,thereby greatly reducing the hardware cost of the entire system.In addition,pure visual design facilitates research on visual algorithms.The visual place recognition algorithm proposed in this paper is applied to the system for real-time testing,which verifies the effectiveness and superiority of the algorithm proposed in this paper,and also verifies that the unmanned vehicle system is reasonable in design,flexible in applicable scenarios,and stable in performance.The unmanned vehicle system is low-cost,simple and easy to implement,and easy to develop,which can provide useful reference and reference for the research of related personnel in the field.