Reinforcement Learning Control For A Magnetic Levitation System

Magnetic levitation technology is a mechatronics technology that combines various high-tech technologies,which is applied to high-end technology fields such as military and aerospace at the early stage.With the continuous development and improvement of control theory,magnetic levitation technology has gradually developed in general industries such as transportation,electrical appliances and materials,etc.The magnetic levitation system has many advantages such as noiseless,pollution-free,and low energy consumption,which attracts much attention from scholars.The magnetic levitation system is a typical nonlinear system,the accurate model of which is necessary if traditional control theory is applied.The Q network is a model-free reinforcement learning method.When the model is unknown,the reward function is used as feedback,and the optimal strategy is found through iteration.In this thesis,the magnetic suspension ball is taken as the controlled plant,and the Q network method is used to design the control system.Firstly,the thesis analyzes the physical properties of magnetic levitation system.By constructing dynamic equations,the non-linear model of the system is obtained.Linearizing the dynamic equations at an operating point,a linear model of system is obtained.The thesis applies linear control theory to the system,and simulation results show that traditional control theory can ensure the stability of system.Steady state performance can be improved by applying integrator to the system.However,problems such as poor adaptability of the traditional controller and parameters of the system obtained uneasily always exist.Therefore,a reinforcement learning algorithm,Q network,is proposed,which can control the magnetic levitation system in continuous state space.A neural network,instead of Q table,is used to generate Q value.Experience Reply,proposed in DQN algorithm,is applied in Q network,which helps reduce the instability of neural network training.The convergence of neural network is improved by setting the target-value network and estimated-value network.Furthermore,traditional reinforcement learning can't evaluate the training data.An evaluation index of steady state is introduced in the Q network to solve the problem that the neural network converges to the local minimum caused by the bias of the training data.Experiments show that the steady-state error is reduced,and control accuracy is improved after the evaluation index is applied to the magnetic levitation system.Finally,numerical simulation of the magnetic levitation system is carried out by using the training results.The results of simulation show that the Q network has good control effect on the magnetic levitation system and can effectively suppress the disturbance.The Q network reinforcement learning controller overcomes the dependence of the traditional controller on the model information and realizes the model-free control of the magnetic levitation system.