Study On Obstacle Avoidance Of Hexapod Robot Based On Transfer Reinforcement Learning

Compared with other mobile robots,hexapod robots have the advantages of having more redundant degrees of freedom and being able to adapt to more complex terrain when dealing with earthquake rescue and rescue tasks under unstructured road conditions in special environments.Traditional robot control methods,such as preprogramming and teleoperation,need to artificially determine the robot's avoidance method.Mobile robots cannot use their own obstacle avoidance experience to prioritize obstacle avoidance strategies in the process of obstacle avoidance,and there are situations where task workload is complicated and work efficiency is low.As a result,mobile robots cannot be used in scenarios with multiple tasks and more complex environments.Therefore,designing a migrating hexapod robot autonomous obstacle avoidance algorithm has important research significance.When the hexapod robot uses traditional algorithms to avoid obstacles,the results are prone to problems such as local optimization and the models cannot be transferred and learned between different obstacle avoidance tasks.In this paper,a research on autonomous obstacle avoidance of a hexapod robot based on transfer reinforcement learning is carried out.In order to realize that the hexapod robot can perform autonomous obstacle avoidance tasks in an environment with multiple types of obstacles,the main research contents carried out in this paper are as follows:(1)This article analyzes the foot-end workspace of a hexapod robot.An obstacle avoidance environment with multiple types of obstacles is designed based on the work space at the end of the hexapod robot.The autonomous obstacle avoidance model of hexapod robot based on deep reinforcement learning and the state space expression method of hexapod robot based on the framework of Double Deep Q Network(DoubleDQN)are studied.An end-to-end discrete action space solution strategy is designed to allow the hexapod robot to perform correct obstacle avoidance actions in the local environment according to the characteristic information of the obstacle feedback.(2)Training models for traditional deep reinforcement learning algorithms can easily lead to overestimation of the state action value,and the sparse positive rewards make it difficult for the model to collect samples with high training value.In this paper,a reward function based on potential energy is used to motivate the hexapod robot to complete the target task,and the random priority sampling method is combined to increase the probability of samples with high training value.By training the hexapod robot obstacle avoidance model based on the improved Double-DQN algorithm in this article,the training effect of the model is compared with the traditional DQN algorithm and Double-DQN algorithm.(3)For the hexapod robot to directly perform obstacle avoidance training in the real environment,there will be situations such as low data sampling efficiency,and irreversible damage to the parts caused by collisions between the prototype and obstacles.This paper uses the Progressive Neural Network(PNN)in migration learning to realize the multi-environment migration of the model.The experiment uses the robot autonomous obstacle avoidance model trained based on the Double-DQN algorithm as the pre-training model,and organically combines different pre-training models into a progressive network structure to complete the migration of obstacle avoidance strategies from the source task to the target task.Then the PNN structure trained in the simulator is transplanted to the hexapod robot prototype for testing.The final test results verify the effectiveness of the hexapod robot autonomous obstacle avoidance method designed in this paper.