Gait Analysis Of Quadruped Robot Based On Deep Reinforcement Learning

Compared with wheeled and tracked robots which are more suitable for flat terrain,legged robots have great advantages in complex environments and rough terrain.Free choices of foot placements and walking gaits allow the legged robots to easily cross some relatively small obstacles,which enable legged robots to be applied to more fileds,such as climbing stairs,jungle search and rescue,exploration of unknown areas.With relatively simple structure,flexible motion control,great stability and excellent dynamic ability,Quadruped robot has become a hot topic in legged robot.Modular controller design is the most common method for legged robot control systems,which divide the overall control tasks into different sub-modules and provides the required data for the next module.However,the manual debugging process of the control parameters in the modular controller will take a lot of time and effort,which can be solved by data-driven methods,like reinforcement learning(RL).Based on the policy search algorithm and value function,reinforcement learning can automatically adjust parameters and correct errors,which can overcome the limitations of modular control.In order to improve the performance of quadruped robot,the thesis introduces reinforcement learning into the control of quadruped robot.The main contents of this thesis are as follows:1.Simulation modeling and kinematics analysis of quadruped robot.The virtual model of quadruped robot is built on the PyBullet simulation platform.The D-H method is used for kinematics and inverse kinematics analysis,which lays a foundation for gait planning and motion control of quadruped robot.2.Design of reinforcement learning controller.The state space,action space and reward function are designed.According to different reinforcement learning algorithms,reinforcement learning controllers are designed.In order to improve the learning performance of algorithms,a fuzzy control system is added to the calculation of the reward function to obtain the relationship between the reward value and the forward speed of the quadruped robot3.Gait analysis of trot based on reinforcement learning controller.The reference trajectory and reinforcement learning action space are integrated to plan the motion trajectory.The quadruped robot is trained by PPO,A2C,DDPG algorithms,and the stable trot gait is realized.The three learning algorithms are analyzed and compared from the aspects of motion stability,reinforcement learning optimization target and energy consumption,which shows that DDPG algorithm can make the robot obtain better motion performance.The simulation results show that the reinforcement learning controller can obtain stable trot gait,and can be applied to the motion control of quadruped robot.