Research On Motion Planning For Hexapod Robot Using Deep Reinforcement Learning

Compared to tracked,wheeled,and other legged robots,hexapod robots have diversified gait,higher stability,and better adaptability to terrain for the movement of robots in complex unstructured environments.It is a challenging task for hexapod robot to plan its footholds in unstructured environment.Traditional motion planning methods use periodic gait or plan gait and footholds separately and treat them as the single-step optimal process.However,such processing causes its poor passability in the unstructured environment with sparse foothold.At the same time,most deep reinforcement learning methods directly train the motion policy,which will lead to a large number of invalid iterations,thus reducing the efficiency of robot motion planning in the environment with dynamic obstacles.Hence,studying a motion planning algorithm for a hexapod robot that can deal with complex unstructured environments is of great significance for improving the passability and motion planning efficiency of the hexapod robot,and giving full play to its motion performance.The plum blossom pile environment is a typical unstructured environment.By changing the number and distribution of plum blossom piles in the environment,it has the ability to represent any unstructured environment in a real scene.Therefore,this paper obtains four typical plum blossom pile environments by changing the distribution of plum blossom piles:a planar plum blossom pile environment,a one-way stepped plum blossom pile environment,a three-dimensional plum blossom pile environment,and a trench plum blossom pile environment.Two new motion planning algorithms based on deep reinforcement learning and free gait are proposed to realize motion planning for hexapod robots in different plum blossom pile environments.The specific research contents are as follows:(1)For the motion planning task of the hexapod robot under a trench plum blossom pile environment,the state transition feasibility model under free gait is established based on the simplified dynamic model,which is used to judge the state transition feasibility.Then,a rule-based free gait planner is proposed to determine the ideal gait sequence in the target state.Finally,a motion planning algorithm based on Proximal Policy Optimization(PPO)and free gait is proposed to realize the motion planning of the hexapod robot in the plum blossom pile environment;(2)Further consider the motion planning task of the hexapod robot in a planar,one-way stepped and three-dimensional plum blossom pile environment with dynamic obstacles.The task of free gait motion planning in complex environments is structurally decomposed into path planning in discrete state space and gait planning and trajectory optimization in continuous state space.The path planning problem of the hexapod robot is regarded as a grid search problem.The real plum pile environment is simplified to a three-dimensional grid environment.The path planning is carried out using the Soft Deep Q-Network(SDQN)based on maximum entropy to obtain the global prior path information.Then the ideal gait sequence under the target state is obtained based on the improved free gait planner,and then the motion trajectory of the hexapod robot is optimized using PPO.Finally,a Hierarchical Free Gait motion planning algorithm based on Deep Reinforcement Learning(HFG-DRL)is proposed,and the motion policies of the hexapod robot in different types of plum blossom pile environments are obtained through training.To verify the feasibility and effectiveness of the proposed algorithm,experiments were carried out in various types of simulation and physical scenes.The experimental results show that the motion planning algorithm proposed in this paper optimizes the motion trajectory of the hexapod robot,promotes the hexapod robot to avoid obstacles quickly and smoothly from the starting point to the target area,and automatically adjusts the gait pattern to deal with different plum blossom pile distributions.The overall planning efficiency of the hexapod robot in the plum blossom pile environment is improved.