Walking Control Of Biped Humanoid Robot Using Reinforcement Learning

Biped humanoid robot has always been popular in robotics. Not only because humanoidhas an appearance resembling human being, but also because of biped walking.This thesis introduces walking control methods based on reinforcement learning. Weproposed a Q learning framework based on RBF network, a.k.a RBF-Q Learning, forcontinuous state and action spaces. By using this framework, we designed two biped walkingstability control algorithms. The main work includes:(1) The design and implementation ofthe new humanoid platform- The Vitruvian Man. In the project, we built a new humanoidrobot hardware platforms and the corresponding software platform. Also we built a simulationenvironment for the humanoid. And we studied kinematics and dynamic of the humanoid.The basic biped walking gait of the humanoid was calculated offline by using Zero MomentPoint theory, a.k.a ZMP, and three dimensional inverted pendulum model.(2) The design of Qlearning framework based on RBF network. In terms of reinforcement learning, we designeda Q learning framework based on RBF network. The framework, which can handlescontinuous states and actions, has excellent generalization capabilities. By using thisframework, we have designed a self-adaptive PID controller. And the effectiveness of RBF-QLearning framework and the controller were validated through an experiment.(3) The designand implementation of two novel biped walking control algorithms based on reinforcementlearning. Using the Q learning framework based on RBF network, two biped walking stabilitycontrol algorithms were designed. One is based on a PID model, and the other is based on aonline action adjusting model. These two algorithms were validated in simulationenvironment. We analyzed the experiment and applied the online action adjusting controller tothe real humanoid robot. The humanoid successfully completed the walking test, whichsuggested the algorithm worked well in real environment.