Research On Application Of Reinforcement Learning In Swing-up And Balance Control Of Inverted Penduum

Inverted pendulum system is a multivariable,high-order,nonlinear and strong coupling of natural unstable system,only use effective control method can make its stebility near the equilibrium position.Many of the key issues in control,such as stabilization,robustness,nonlinearity,and tracking problems,can be reflected in the control of an inverted pendulum.At the same time,the biped robot to walk,satellite's and the rocket's posture adjustment are similar to the inverted pendulum systenm,So,the research of inverted penduum system has important theoretical value and important engineering practical significance.The traditional inverted pendulum control methods such as RID control and LQR control are based on model control,but the inverted pendulum system is difficult to obtain a precise mathematical model.Reinforcement learning,as a kind of machine letarning,differs from supervised learning in that it requires a teacher signal,It emphasizes that an evaluation feedback signal is obtained during the interation with the environment,and the evaluation infonrmation is used to optimize the behavioral decision,There is no need to know the system model,which avoids errors due to modeling.Therefore,this paper applies reinforcement learning to the pendulum swinging and balance control tasks,and focuses on the application of reinforcement learning algorithms in the wntrol of inverted pendulum balance.The main results of this article are twofold.For the control of swing control,this paper studies and implements the inverted peandulim swing control based on the Q learning algorithm.The-first-order inverted pendulum simulation control experiment show the effectivenes of te Q-learning algorithm in pendulum pendulum swing control.In terms of balance control,this,paper has studied the Least Square Policy Iteration and Kemel-based LSPI iterations to effctively solve inveried pendulum balance,Based on this,in view of the poor approximation ability and generalization ability of LSPI algorithm and the high computational complexity and computational cost of KLSPI algorithm,an LSPI alagorithm based on Extreme Learning Machine was proposed.The algorithm,while improving the accuscy and generalization ability of the traditional LSPI algorithm,controls its computational cost.The first-order inverted pendulum simulation control experiment shows that KLSPI and ELM-LSPI algorithm bothimprove the convergencc ability and generalization ability of LSPI algorithm to a certain extent,but the computational cost of ELM-LSPI algerithm is less than KLSPI algorithm,it's more conducive to subsequent physical research and online algorithm development.