Research On Multi-gait Locomotion Control Of Quadruped Robots Via Deep Reinforcement Learning

In order to strike a better balance between the need for speed and minimum energy consumption,quadrupeds have developed multi-gait patterns through longterm evolution and can smoothly transition from one gait to another when necessary.This ability allows them to be more flexible and efficient when crossing natural terrains,and also reduces energy consumption during long-distance travel.In addition to their flexible movement abilities,quadrupeds are also able to navigate complex and changing natural terrains with ease.As a type of automated device inspired by quadrupeds,quadruped robots face two challenges in practical application:(1)How to reduce energy consumption and accomplishment versatile locomotion?(2)How to implement robust navigation through complex terrains?Traditional model-based methods have provided different levels of solutions to these two problems and have achieved good results.However,due to the non-linear characteristics of the quadruped robot model and the complexity of the external environment,the effectiveness of these methods is often limited by the domain expertise and debugging experience of experts,the perception ability of the robot,and the computing power of the control platform.In recent years,a class of data-driven methods,represented by deep reinforcement learning,has provided a new approach for automatic design of robot locomotion control strategies,without requiring precise robot modeling,greatly simplifying the design process,and demonstrating good adaptability to real terrains.However,due to the limitations of the reward function or prior knowledge pre-given by artificial settings,locomotion control methods via deep reinforcement learning are often restricted to a single gait.In response to the multi-gait versatile locomotion problem of quadruped robots,inspired by the relationship between quadrupedal gaits and energy consumption,we proposed a deep reinforcement learning-driven multi-gait locomotion control stability method for quadruped robots.The method divides the robot’s motor rotation angle command into two parts:(1)periodic signal: by designing a novel explicit gaitinspired trajectory generator,the periodic signal is generated to guide the production of multi-gait behavior and accelerate the training process;(2)feedback signal: using the feedback signal generated by the neural network optimized by deep reinforcement learning to adjust the robot’s state in real-time,so that the robot learns to reach highprecision speed commands and low energy consumption,the two performance objectives of locomotion.In response to the problem of quadruped robots’ ability to navigate complex terrains,we creatively construct a teacher-student learning framework for multi-gait stable locomotion based on the embeddable property of the proposed multi-gait locomotion control method and the information compression and extraction properties of neural network encoders.This is a learning paradigm of knowledge distillation.With this method,the paper was able to use a neural network locomotion controller to make quadruped robots simultaneously master flexible multi-gait motion abilities and stable complex terrain navigation abilities for quadruped robots.To verify the effectiveness of the proposed multi-gait locomotion control method and teacher-student learning framework,we conducted a large number of experiments in the Pybullet simulation environment and on a real parallel quadruped robot platform.The experimental results show that the proposed multi-gait locomotion control method enables quadruped robots to generate multi-gait behavior patterns and automatically implement smooth transitions between different gaits based on speed commands,with higher energy-consumption efficiency.The constructed teacherstudent learning framework enables quadruped robots to effectively enhance their ability to navigate complex terrains while maintaining multi-gait locomotion capabilities.In addition,in experiments conducted in a real environment,the parallel quadruped robot also demonstrated capacity of resisting disturbance and robustness.It not only effectively resisted random environmental disturbance but also successfully navigated through terrains such as snow,grass,steps,uneven terrains and so on.These results demonstrated the feasibility and effectiveness of the proposed method.