Designing And Reinforcement Learning Controlling Of A Quasi-passive Dynamic Walking Robot

Passive dynamic walking is an important research topic in the field of bipedrobot. The goal of passive dynamic walking research is to discover the essentialcharacteristics of biped walking. The research methodology includes gait synthesisand gait analysis. In gait synthesis, the characteristics of walking are learnedthrough developing of passive walking robots and controlling them to walk. Inthis thesis, a 2D quasi-passive dynamic walking robot is built and controlled toachieve robust and energy e?cient walking by the actuating of compliant actuatorswhich is similar to human muscle-skeleton actuators. The main contributions ofthis thesis include:1. A numerical algorithm is presented to optimize the mechanical parametersof the robot. The algorithm is based on the stability analysis method ofnonlinear dynamical systems. Since the model is similar to the structure ofthe real robot, the conclusions can be applied in the designing of the robot.2. Based on the research of the compliant actuating methods, a type of com-pliant actuator is selected for the quasi-passive dynamic walking robot.3. A 2D quasi-passive dynamic walking robot is designed and built. Therobot is actuated by MACCEPA compliant actuators to imitate the muscle-skeleton actuators in human walking. With the aid of compliant actuators,the joint sti?ness and joint torque of the robot are controlled independentlyduring walking. Compared with most quasi-passive dynamic walking robots,this robot can achieve more human-like gait. However, the dynamic walkingcontrol of the robot is more complicated.4. A reinforcement learning based method is proposed to control the robotto walk. The task is divided into two sub-tasks: the learning of referencegait and the controlling of walking in unideal environment. A walking gaitmodel based Q-learning method is presented to solve the first sub-task. The dimension of the state-action space is descended by incorporating of theprior knowledge with the model. A fuzzy advantage learning based methodis presented to solve the second sub-task. The generalization ability of thefuzzy inference system is used to control the robot to walk in environmentwith disturbance or model uncertainty. The advantage parameter sets ofthe learning controller are well initialized by a numerical algorithm. In thisway, the e?ciency of the learning controller is improved.