Humanoid Robot Balance Control Through Arm Swing Via Reinforcement Learning

At the moment,the researchers focus on the lower limb in research of robot balance control,but less attention is paid to robot hand movement.Obviously,the inertia of swinging arm can have a considerable impact on the balance of the robot,and if using the traditional mathematical method,first we need a large number of physics knowledge,and the model may be not accurate or universal.But using reinforcement learning to make robot learn balance control automatically is not only the trend of robot balancing algorithm,but also has good adaptability to hardware.Therefore,this paper attempts to learn robot arm control algorithm by machine learning,in order to achieve the task of balancing the robot’s balance with the inertia generated by swinging the robot arm.In order to simplify the problem,this paper do not consider the other joints of the robot,considering only the robot shoulder joint,which let the robot tilt in the longitudinal direction,then to achieve balance by swinging arm.First of all,using Q-learning algorithm to generate a single inverted pendulum control algorithm,which helps to understanding the learning algorithm,the model is used to divide a tabular design method based on the state of the inverted pendulum,and use Simulink for simulation.Secondly,using the sarsa learning algorithm to generate another single inverted pendulum control algorithm,and simulated by Simulink,and compare it with the Q-learning algorithm,the Q-learning algorithm is easy to converge.Again,useing the Q-learning learning algorithm for machine learning in simrobot simulation environment to study the control algorithm of controlling the robot in the upright condition through arms swinging back and forth.To keep the balance of the inverted pendulum model is similar to the problems in front,the differents is that it is through the swing arm to reach balance,and has more complex state space and action set,although the robot has reduced to only 2 joint,but the state space is still very large,learning obtained relatively satisfactory results.Finally,using the Q-learning algorithm,constructs a control algorithm to adjust the arm of the moving robot to balance the robot,although experiments did not get good results,but the simulation experiments show the effectiveness of the control algorithm,but also reflecting that when the state space and action set expand,the learning effect of Q-learning will decrease.