Research On Deep Reinforcement Learning Methods For Autonomous Control Of Mobile Robot

With the advent of the artificial intelligence era,robotics and artificial intelligence technology have become the hotspots of current research.How to make machines more intelligent and autonomous has become a major topic of current research.The navigation and autonomous control technology of the mobile robot have always been its important research direction.At present,the traditional method is to map the whole environment and then locate itself,and finally carry out path planning and track the trajectory.Such an approach is more complicated on the one hand which needs a lot of artificial design work,and does not conform to the way people think in the other hand.It is a question worth studying to make the robot have the ability to self-learn and self-explore,so that it can think like human beings that just need to see the scene in front of it and know how to make decision.Robots interact with the environment through deep reinforcement learning,which makes the robot capable of self-exploration and self-learning.By inputting the currently seen scene image,it is possible to output command-level navigation commands end-to-end without the map of environment.This method is similar to how humans remember and navigate the scene.Through continuous trial and error,using reinforcement learning to maximize the benefits of action,agent can learn how to act when encountering similar scenarios.Deep reinforcement learning requires a lot of time and unnecessary failure attempts.Therefore,this paper combines the deep reinforcement learning with imitation learning,and proposes a deep imitation reinforcement learning algorithm.The agent first learns from the teaching experts which can improve the efficiency of exploration,and then continues to improve performance through self-learning after reaching the level of experts.In the AI2-THOR simulation environment,we use this algorithm to carry out end-to-end autonomous navigation experiments of indoor robots.Then,end-to-end control of unmanned vehicles is carried out in Mario racing environment and PreScan simulation platform.The effectiveness of the algorithm is verified by comparing with traditional deep reinforcement learning.The main research results in this paper include:1.A deep imitation reinforcement learning framework which combines the deep reinforcement learning and imitation learning is proposed.Aiming at the shortcomings of deep reinforcement learning,such as long learning time and low efficiency of exploration,it uses expert experience data to guide reinforcement learning to make the agent accelerate learning efficiency,and at the same time,it can continuously self-learn on the basis of expert data.The algorithm first trains the expert strategy network through imitation learning,provides efficient and excellent training samples for deep reinforcement learning through the expert network,and affects the reward function.The training samples of deep reinforcement learning can also become training samples for the expert network to continue optimization.2.A deep imitation reinforcement learning algorithm based on the A3C(Asynchronous Advantage Actor-Critic)algorithm is proposed.The experiment of deep imitation reinforcement learning is carried out in the indoor simulation environment THOR.In the simulation environment,the robot observes the image as the input,and achieves a better end-to-end autonomous navigation effect through the deep imitation reinforcement learning.In this paper,the new algorithm and the classical deep reinforcement learning algorithm are compared,and the effect of the new algorithm is greatly improved.What's more,the speed of learning is accelerated and the path to the goal is better.3.A deep imitation reinforcement learning algorithm based on the DQN(Deep Q-learning)algorithm is proposed.The autonomous control of mobile robot is studied by using deep imitation reinforcement learning in the Mario racing game and the prescan environment.Unmanned vehicle can be controlled autonomously to follow the lane only through the image information observed from the first view angle.Compared with the traditional DQN algorithm,the algorithm has faster learning speed and better performance.