Research On Locomotion Control And Sim-to-real Of Quadruped Robot Based On Deep Reinforcement Learning

In complex environments,the suspension structure and the form of discrete landing points of the quadruped robot make it have the potential to explore all terrain on earth like animals,and it is considered to be the best mobile platform for field exploration,post-disaster rescue,patrol and search and other tasks.The structure of quadruped robot is complex,and the traditional locomotion control requires accurate kinematics modeling and tedious manual tuning,so it is impossible to find the optimal control strategy to cope with the terrain environment,and the real-time adjustment ability is poor.However,the locomotion control based on deep reinforcement learning has the advantages of no model,strong applicability and autonomous strategy generation.However,the learning control policy usually stays in the simulation stage,and it does not perform well in the actual robot due to the difference between simulation and reality.Based on the above difficulties,this paper builds the physical platform of the quadruped robot,designs the locomotion control algorithm,realizes the locomotion control of the quadruped robot based on deep reinforcement learning in the simulation,and completes the traditional locomotion control and the control policy transfer based on learning on the physical object.The specific content of the paper is as follows:In response to the lack of physical verification of control algorithms and strategies,the quadruped robot physical platform was built.Control the mass and size of the robot from the single leg and the overall structure,increase the effective torque while maintaining low inertia and high sensitivity;in terms of control system,improve the motion of the quadruped robot from the four aspects of power system,perception module,main control module and power system.performance.Aiming at the problem of the lack of control algorithms to make the quadruped robot's physical platform move stably,a motion control algorithm for the quadruped robot is designed.First,perform kinematics modeling of the robot,including the overall robot model,single-leg forward and inverse kinematics,and single-leg Jacobian matrix.Then,by analyzing the gait timing of different motions,the foot trajectory is planned separately for the swing phase and the support phase.Finally,the virtual model is used to control the current required to convert the foot end position into the motor to drive the robot to move.In view of the problem that traditional control methods require complex and cumbersome manual tuning and cannot find the optimal control strategy for the terrain environment,in the simulation environment,the use of the SAC algorithm in deep reinforcement learning can well weigh the advantages of exploration and utilization,based on SAC Algorithm design simulation environment and deep reinforcement learning model,on flat ground,realize the motion control of quadruped robot based on deep reinforcement learning.For simulation and realistic physical differences make the problem difficult to complete the migration process,and analyze the reasons for the differences,using a simulation model to improve the matching of training and more robust control strategies to narrow the gap between simulation and reality,using the reward function To measure the control performance,in the actual quadruped robot test,complete the traditional motion control and the migration of the control strategy based on learning.