Study On Control Method Of Legged Robot Based On Reinforcement Learning

The legged robot has a wide range of application scenarios due to its flexibility,but its motion control is relatively complex.The traditional method requires very strict accuracy of robot hardware and software,resulting in high cost.Therefore,this thesis hopes to make use of the algorithms in the field of deep reinforcement learning(DRL)so that robots can find control methods suitable for their own structures through learning.In this thesis,the task-oriented robot framework was obtained by combining the reinforcement learning algorithm with the robot,and the robot walking task model was proposed within the framework.Then the reinforcement learning algorithm was further studied to improve the performance.Finally,the experimental environment was built and tested.The main work can be summarized as follows:(1)Advanced algorithm SAC(Soft actor critic)was improved.The segmented floating adjustment factors are used to replace the fixed adjustment factors,and the proportion of policy entropy in the objective function is dynamically adjusted during training.The simulation results show that the performance of the improved SAC algorithm is improved by 7.7%compared to the original SAC algorithm.(2)A new DRL algorithm SMPSAC was proposed.First,the model predictive control(MPC)is used to optimize the output of the improved SAC algorithm to obtain the intermediate algorithm MPSAC.Then the policy entropy is used to improve the objective function of the MPC,weaken the negative effects brought by the model errors in the MPSAC,and obtain the final algorithm SMPSAC.The simulation results show that the performance of the algorithm SMPSAC is improved by 20%compared to the original SAC.(3)A robot test platform was Designed and built.The platform includes:a 4-legged robot assembled using 3D printed components,a robot state detection device and a reset device.The test platform meets the requirements of the DRL algorithm,the robot and learning targets can be easily replaced.(4)Four-legged robot walking test.Real-world robots learn to walk using the SMPSAC algorithm.This thesis not only proposes a new DRL algorithm,but also proves in real scenarios that as long as the goal is set and a proper reward is given,the robot can continuously progress towards the goal,providing a kind of low-cost foot-type robot motion control solution.At the same time,in the future,the goal of robot learning can be changed without changing the DRL algorithm and experimental environment,which simplifies the design process of the robot.