Research On Gait Control Strategy Of Quasi-passive Biped Robot Based On Deep Reinforcement Learning

Biped robots have better mobility than other types of ground mobile robots.And biped robot are the best carrier of service robot,and they're also the easiest types of robots that can be introduced to people's life.The quasi-passive biped robot have lower energy consumption and natural human-like gait,and become a hotspot in the research of biped robot.However,the quasi-passive biped robot have a hybrid dynamics system,the system is nonlinear,strong coupling and multivariable,make the robot have the problems of walking stability and walk control.Walking stability as the most important indicator to measure walking performance of quasi-passive biped robot,and walking stability is the key to robot application.Therefore,how to control the walking of the quasi-passive biped robot is a critical issue that needs to be resolved.As a new emerging artificial intelligence technology,Deep reinforcement learning have Possess strong perception ability and decision control ability.Deep reinforcement learning have important research value for the walk control of quasi-passive biped robots.In this paper,for the problems of walking stability and walk control of quasi-passive biped robots,this paper propose a walk control method for quasi-passive biped robots based on deep reinforcement learning.The main research work of this paper is as follows:First,through the analysis of the walking process of the biped robot with the forward angle feet to establish hybrid dynamic model,and introducing Poincaré mapping method to analyze the robot's walking stability.The Newton-Raphson iterative algorithm is used to obtain the fixed point of the biped robot,and analyzed the influence of the biped robot's parameters on the walking stability.It laid the foundation for future research on walk control methods.Secondly,in order to improve the robot's walking stability,combined with the characteristics of biped robot walking to establish agent states,agent actions and reward functions.Based on learning efficiency and final learning effect,proposes a biped robot walk control base on Deep Deterministic Policy Gradient and a biped robot walk control base on Proximal Policy Optimization.Realize the effective control of the biped robot walk under the fixed slope.Thirdly,in order to improve the stable walking ability of the robot on different slopes and improve the learning ability of Deep Deterministic Policy Gradient.Base on Deep Deterministic Policy Gradient proposes Ape-X DPG,and redesign the agent state,reward function and episode process.Realize the control of the robot's stable walking motion on different slopes,and through walking stability analysis to verify Ape-X DPG control ability of robot walking.Finally,use robot model parameters as part of the state of the agent,giving deep reinforcement learning methods the ability to autonomously identify different biped robot,and propose a general gait control method based on deep reinforcement learning.Realize the control of the walking movement of different robots on different slopes.In the simulation,through Proximal Policy Optimization with biped robot model parameters,realize efficient and stable control of the gait of three different structures and parameters of the robot.