Research On Inverted Pendulum Control Algorithm Based On Reinforcement Learning

In the age of technology,artificial intelligence is ubiquitous in all areas of our lives,from AlphaGo to AlphaZero.As the core of artificial intelligence,machine learning is the fundamental way to make computers intelligent.As a hot direction in the field of machine learning research,reinforcement learning Iearns from the interaction between the agent and the environment,and continuously updates and improves the control strategy in a self-learning way to gradually achieve optimal or near-optimal control effects.Because reinforcement learning is a model-free and unsupervised machine learning method,it has the advantages of strong versatility,wide application range,self-tuning of parameters,etc.,which can greatly reduce the design difficulty and manpower input of the control system,and has broad application prospects.Therefore,the study of reinforcement learning has important theoretical value and practical engineering application value.The inverted pendulum system is a multivariable,nonlinear,high-order,strongly coupled self-unstable system that can simulate most common control objects.Its control algorithm has the characteristics of multiple input and single output,so it is a typical automatic.Control theory research device.Taking this problem as the research object,it can effectively reflect the follow-up,robustness,tracking and stabilization of the control algorithm in the practical application system.Therefore,this paper takes the first-order linear inverted pendulum system as the object and studies the deep reinforcement learning algorithm.The main research contents and results are as follows:(1)Introduce and analyze the basic concepts of reinforcement learning.Through the theoretical derivation of the commonly used reinforcement learning algorithm and the analysis of the parameters of the Markov decision process,the theoretical basis for the reinforcement learning and the application of the deep reinforcement learning algorithm in the inverted pendulum control system is laid(2)Through the OpenAI Gym game library,the experimental simulation of deep reinforcement learning DQN algorithm,dual network DQN algorithm and PG algorithm in the balance control of the first linear inverted pendulum is completed.The results show that all three control algorithms can complete the training quickly and achieve the balance control of the inverted pendulum.On this basis,the paper further studies and tests the effects of three different reward methods on the algorithm.Through experimental comparison,it is found that the linearized reward method has the fastest training speed,which provides a reinforcement learning for real inverted pendulum control training.Important experience(3)The hardware experiment platform of the first-level linear inverted pendulum based on PLC is built.The balance control of the inverted pendulum is verified by the PID control algorithm.The validity of the experimental platform is verified and the test benchmark is provided for the control research of the reinforcement learning algorithm(4)Based on the DQN algorithm,the swing control of the first-order linear inverted pendulum is realized.After about 100 training rounds,the control algorithm can swing the pendulum from the sagging position to the upright position within 200 control cycles(one control cycle is 20ms)to complete the inverted pendulum swing control(5)Based on Q-learning,the inverted pendulum balance control is studied.Aiming at the limitations of real control environment,such as high noise,limited training and difficult to obtain some training samples,an off-policy control algorithm with multi-training strategy is designed.It can be obtained from artificial teaching or other control algorithms.Experience,improve training efficiency,and thus quickly complete training and obtain better control effects through limited experiments.Because the method effectively reduces the amount of experimental training,so that the reinforcement learning algorithm can achieve better control effects with less manpower input,and has made preliminary practice for the application of reinforcement learning in practical engineering,and has a good application prospect.