Research On Reactive Obstacle Avoidance Using Deep Reinforcement Learning And Transfer Learning

In recent years,as the CPUs has become more powful,and the capacity of memory gets larger and larger,and intelligent mobile robots represented by unmanned vehicles and drones are getting closer to our lives.When perform tasks in the scene,the most basic capabilities of mobile robots is to avoid obstacles.Traditional non-machine learning obstacle avoidance algorithms require developers to construct three-dimensional structures of the scene or manually adjust a large number of parameters.At the same time,it is impossible to use the experience of obstacle avoidance to self-itate in the process of avoiding obstacles.Most of the methods based on convolutional neural networks model the obstacle avoidance problem into a classification problem as supervised learning,and the label of each image needs to be manually de-labeled,which is time-consuming and labor-intensive.The obstacle avoidance algorithm based on deep reinforcement learning does not need to reconstruct the complex scene in three dimensions,and directly models the obstacle avoidance problem into a decision process and can output the action end-to-end.However,there is still a problem of insufficient generalization ability for strange scenes,and it takes a long time to fine tune the network in a strange scene.This article is devoted to the study of a reactive obstacle avoidance system that can be applied quickly and accurately in different scenarios.In the aspect of reactive obstacle avoidance,this paper first proposes an improved algorithm based on Ego Dynamic Space Transform(EDST),which uses the depth map of the monocular depth estimation as input to select the optimal waypoint at the next moment.And then this paper uses the Double DQN algorithm to build an end-to-end system,using the observed depth map of the robot as input,and directly outputting the executed action and compare the obstacle avoidance effects of the above two algorithms;for different scenarios,this paper adopts The idea of the Generative Adversarial Networks(GAN)can be achieved by mapping the network to map the features of the target scene to the feature space of the source scene,and using the decision network of the source scene to implement the output of the action commands.In the end,the robot can immediately avoid obstacles in different scenes.After experimental testing,the algorithm can guide the robot to avoid obstacles properly.At the same time,for the unfamiliar scene,the transfer learning algorithm proposed in this paper consumes less training time than the fine-tuning network under the same obstacle avoidance success rate.We also built a UAV flight obstacle avoidance system in the real scene,which proved the practical value of the algorithm.