Research On Deep Learning Based Visual Place Recognition

After long-term development,robot technology has become an indispensable part of society.At present,robots have become an essential tool for improving production efficiency and reducing labor costs.With the increasing problem of social aging and soaring labor costs,robots are receiving increasing attention from academic and industrial circles.With the proposal of China's 2025 plan and the country's attention to artificial intelligence technology,the robotics field has begun to flourish,and all kinds of robots have mushroomed into people's lives.As a critical technology in the field of robotics,visual place recognition has also attracted the attention of more and more researchers.Visual place recognition is mainly a question of “Where am I?”.It is a key component in solving many problems in the field of computer vision and robotics,such as semantic-based image retrieval,loop closure detection module in simultaneous localization and mapping(SLAM),visual localization and augmented reality,etc.Visual place recognition mainly faces three significant challenges: the robustness against condition changes,the robustness against viewpoint change,and the efficiency requirements of the algorithm.Condition changes refer to changes in external conditions such as illumination,weather,and seasons changes,which may cause images taken by the camera at the same location exhibit different appearances.The viewpoint changes mean that the images taken at the same place exhibit different appearances due to the different orientation of the camera.Efficiency requirement implies that the visual place recognition algorithm should respond within an acceptable time frame when the robot is operating in a large scenario(such as a city level).In this paper,based on the robustness and efficiency problems of visual place recognition,the algorithm research of visual place recognition based on deep learning is deeply carried out.The main research contents and innovations are as follows:1.Random Error Analysis and Application of Image Representation for Visual Place Recognition(Chapter 2)Visual place recognition is typically modeled as an instance retrieval task that matches the current image with the images in the image database with geo-location tag,and thereby identifies and locates the location.Therefore,the magnitude of the random error of the image representation is a crucial factor affecting the accuracy of the place recognition algorithm.In this paper,the random error of image representation is analyzed for the first time,and the main factors affecting random error are determined.Based on this,a visual place recognition algorithm SeqCNNSLAM which can reduce the influence of random error of image representation is proposed.The experimental results show that SeqCNNSLAM can significantly improve the accuracy of the place recognition algorithm compared to other visual place recognition algorithms.Also,to further enhance the representation ability of the image representation vector to the environment,especially for the valuable information such as landmark buildings,we propose a patch-based SeqCNNSLAM(referred to as P-SeqCNNSLAM)based on SeqCNNSLAM.The experimental results show that P-SeqCNNSLAM can further improve the accuracy of the visual place recognition algorithm compared to SeqCNNSLAM.2.Retrieval Scope Constraint Method Based on Physical Spatial and Image Representation Spatial Topological Relationship(Chapter 3)As the robot runs,it will acquire increasing images.When the robot performs place recognition,if the range of candidate matching images of the query image in the database cannot be properly constrained,the number of candidate matching images will be larger and larger.This will increase the amount of computation required by the system to identify the corresponding location of an image so that the response time of the algorithm is difficult to meet the efficiency requirements.Aiming at the above problems,this paper studies the topological relationship between the images acquired by robots in physical space and image representation space,and proposes a visual place recognition algorithm(referred to as A-SeqCNNSLAM)that can constrain the image retrieval range.Experiments have shown that on a standard dataset,A-SeqCNNSLAM can achieve a time acceleration of about 20 times while making comparable accuracy compared to the SeqCNNSLAM method.3.End-To-End Visual Place Recognition Framework: Image Feature ExtractionFeature Aggregation-Image Representation Compression(Chapter 4)Studies have shown that visual convolution features have higher robustness and generalization performance than traditional hand-crafted features.Although some researchers have proposed some neural networks for visual place recognition,the existing methods fail to meet the requirements of accuracy and efficiency simultaneously.That is to say,the high degree of discrimination of image representation is at the cost of high dimensions,which will greatly increase the calculation amount of image matching and bring great inconvenience to practical use.Following the typical instance retrieval process,this paper proposes an end-to-end learning framework for place recognition based on the convolutional neural network,called NetPR.The NetPR method consists of three modules: feature extraction,feature aggregation,and image representation compression,which takes into account the requirements of algorithm accuracy and efficiency,and can directly generate low-dimensional and high-discrimination image representation.To verify the validity of NetPR,two neural networks NetPR1.0 and NetPR2.0 are constructed based on NetPR framework in this paper.The experimental results show that compared with the existing visual place recognition algorithms,the algorithm based on NetPR framework can greatly improve the accuracy of the algorithm while greatly reducing the image representation dimension,which is an efficient and robust visual location recognition algorithm.4.Design and Implementation Scheme of Visual Place Recognition System(Chapter5)Based on the existing research results and the innovation of this paper,this part takes several common application scenarios of visual place recognition as the entry point and proposes a set of complete system design and implementation scheme to guide the construction of the actual system and the productization of the algorithm.