Visual Obstacle Avoidance And Navigation Research Based On Deep Reinforcement Learning With Fused Spatio-Temporal Features

In recent years mobile robots have been widely used in industrial and agricultural production,storage and logistics,health care and social services.The ability to navigate autonomously is a fundamental and critical technology for mobile robots serving all sectors.Traditional navigation techniques rely on accurate maps of the environment and different hyperparameters that need to be adjusted for different environments,which makes it impossible to migrate to unfamiliar environments.Deep reinforcement learning provides an autonomous learning capability for mobile robot navigation systems,providing an effective solution for extracting features,understanding the environment and controlling decisions in unfamiliar environments.This paper investigates visual obstacle avoidance and navigation for mobile robots based on deep reinforcement learning,and completes the task of obstacle avoidance and navigation for mobile robots in both simulated and real-life scenarios.The main research elements are as follows:(1)For mobile robots equipped with depth camera,visual obstacle avoidance algorithm for deep reinforcement learning based on fused temporal features is proposed in this paper.The method fuses depth images generated by a depth camera and pseudo-Li DAR data converted from depth images to improve the performance of a single sensor for sensing the environment.A long short-term memory network is introduced to capture the temporal relationships between the pseudo-Li DAR data at successive moments to improve the convergence speed.Experimental results show that the method exhibits faster convergence and higher cumulative reward than the single-sensor approach and also shows good obstacle avoidance in real scenarios.(2)To address the problems of insufficient spatio-temporal feature extraction from original images,weak target point-oriented exploration and low generalization ability of visual navigation algorithms based on deep reinforcement learning,a new visual navigation framework is proposed in this thesis.A feature extractor based on convolutional long short-term memory networks is designed,which not only captures the features associated with target point information in continuous moment depth images,but also better captures the key spatial layout and corresponding temporal features from temporal images.The experimental results show that the algorithm achieves good results in terms of learning rate,convergence and navigation success rate and also shows good generalization across scenes.