Research On Motion Planning Of Quadruped Robot Based On Deep Reinforcement Learning

Currently,quadruped robots have become one of the hotly researched legged robots due to their unique advantages.The driving force behind this is the flexible and agile motion,the ability to adapt to complex terrain,the high load capacity,and the wide range of practicality of quadruped robots.Traditional control methods for quadruped robots have been developed to a relatively mature level,but still suffer from cumbersome and complex controller designs and poor versatility.To explore more autonomous,intelligent,and simple motion planning and control methods for quadruped robots,this thesis conducts research on the motion planning methods for quadruped robots based on learning methods,and the main research works are:1.Aiming at the problem of the cumbersome design of traditional motion planning controllers for quadruped robots,this thesis proposes a motion planning controller for quadruped robots based on the deep reinforcement learning method.First,the motion planning problem of the quadruped robot is modeled as a Markov decision process in the deep reinforcement learning problem,and the data such as joint position and pose of the quadruped robot are selected to compose the state and action space;second,a reward mechanism with generality is designed,and the reward function introduces the motion stability and energy consumption index of the robot on the basis of considering the motion speed index,which The reward mechanism not only ensures that the robot can achieve high-speed and stable motion with low energy consumption after training,but also encourages different motion goals by adjusting the weight of the reward term in the reward function;then,a gait motion reference framework is designed as a priori information to guide the robot to learn gait motion,and the reference foot trajectory is planned by the characteristics and parameters of different gaits and substituted into the inverse kinematic model to obtain the reference motion,and the reference motion is combined with the learned motion to achieve the gait motion.The reference motion is combined with the learned motion to realize the motion of the quadruped robot,which improves the learning efficiency of the gait motion of the quadruped robot and the convergence speed of the algorithm;finally,the TD3 algorithm is introduced into the training process of the motion planning strategy of the quadruped robot,and the algorithm converges faster and the generated robot motion is more stable compared with the commonly used PPO algorithm.The feasibility and effectiveness of the proposed method are verified by simulation experiments.2.To address the problem that the deep reinforcement learning motion planning method for quadruped robots does not have generality,this thesis proposes a transplantation method of motion planning strategy for quadruped robots.First,different robot models are introduced in the deep reinforcement learning environment of the quadruped robot,and the descriptions of the robot torso,leg links,and joints are unified between different models;then,the morphology and dynamics of the robot models are randomized during the training process,and the degree of randomization is increased by introducing randomization coefficients to improve the robustness of the motion planning strategy transfer;finally,the strategy transfer is validated in simulation Finally,the strategy transfer is verified in simulation using different robot models,and it is demonstrated that the motion planning strategy can be successfully transferred to different robot models,which verifies the portability of the proposed method.