Visual Place Recognition Based On Hard Example Mining And Domain Adaptation

With the continuous development of computer vision related technologies,simultaneous localization and mapping(SLAM)technology is more and more widely used in human life,such as autonomous navigation robots,automatic driving,drones,virtual reality and so on.But in SLAM,we still face a lot of problems that need to be solved,such as loop closure detection.The task of loop closure detection is to determine whether the current location is a previously visited location.The purpose is to add a new constraint to the build map task,thereby eliminating the cumulative error generated during the map construction process.The essence of loop closure detection is place recognition.Traditional place recognition algorithms,such as the visual word bag model,rely on hand-designed image features that change when changing scenes(changes in moving objects such as pedestrians and vehicles in the scene),viewpoint changes,and differences in day and night illumination.Thereby affecting the performance of the traditional place recognition algorithm.In recent years,deep learning has made great progress in the field of computer vision.Because deep learning based algorithms are end-to-end,the neural network can be trained with specific tasks to automatically extract more robust image features.Therefore,some scholars have proposed many algorithms based on deep convolutional neural network for visual position recognition.The algorithm proposed in this thesis is also based on deep convolutional neural network.Compared with traditional algorithms,it can solve the problem of low recognition rate in the case of changing scenes and viewpoint changes.However,existing algorithms based on deep convolutional neural networks do not fully consider the problem of interference with negative samples similar to the target image to be identified.These negative samples are samples that are difficult to distinguish correctly for deep convolutional neural networks,so-called hard example.In the existing FaceNet network,only difficult samples are selected in the batch inputted into the network,the loss of the neural network on each sample in the current batch is calculated,and the maximum loss is selected as the loss of the batch.Therefore,the FaceNet algorithm can only select difficult cases from the current batch.In addition,in the NetVLAD network,for each set of training samples,only the most difficult negative samples are selected among the 1000 negative samples,but this still cannot find the most difficult sample in the overall training set.Therefore,this thesis proposes an algorithm of hard example mining for the whole training set.After the iterative training of the neural network using the whole training dataset,the algorithm selects the negative samples that are most difficult to distinguish from the neural network from the overall training dataset to form a new training dataset.The negative sample,the new composition of the training set as the next round of training data for the neural network.In this way,the negative samples with difficult training concentration can be fully excavated,and the number of difficult samples participating in the training is more,so that the network model avoids the interference of negative samples similar to the target image to be identified.In addition,due to the lack of sufficient datasets collected at night,existing depth learning based place recognition algorithms are trained using image datasets collected during the day.When the test dataset comes from nighttime when the light intensity is extremely low,the performance of the algorithm drops dramatically.If the training set is regarded as the source domain and the test set is regarded as the target domain,this is a typical classification problem in the case of different source and target domains.Although collecting images taken under different illumination conditions at different time periods as training samples can avoid different problems between the training domain and the test domain,and improve the generalization ability of the network model,it wastes a lot of human resources.Aiming at this problem,this thesis proposes a domain adaptive algorithm based on the confrontation discriminant network and a domain adaptive batch normalization algorithm,so that the neural network has stronger domain adaptive ability.In the domain adaptive algorithm based on the adversarial discriminant network,the adversarial discrimination network is added to the original network,and the image feature distribution of the neural network extracted on the source domain and the target domain image is similar by the training of adversarial discrimination,and the difference of images features caused by the different distributions of the training set image and the test set image are reduced,and the recognition rate of the network model across domains is improved.In the domain adaptive batch normalization algorithm,considering that different image distributions are essentially different mean and variance of distribution,in this thesis,the feature maps in the source and target domains are normalized using the mean and variance of the feature maps of the images in the respective domains.So that the difference of feature distribution on different domains is reduced after normalized,and the domain adaptive ability of the network model is enhanced.