Research On Optimal Control For Nonlinear Systems Based On Reinforcement Learning

The optimal control problem of nonlinear systems is one of the important research topics in the field of control theory.As the strong nonlinear characteristics of the actual systems,the idea of solving the optimal control problem using the traditional model-based control methods is unavailable by establishing the mathematical models of the systems.Therefore,it is very important and valuable to explore the optimal control methods of nonlinear systems when the models are not completely determined or unknown.The reinforcement learning algorithm is one of the effective intelligent control methods when the system models cannot be accurately obtained.Therefore,this thesis is mainly based on reinforcement learning method to solve the optimal control problem of nonlinear systems,and the main research contents of this thesis are as follows:(1)A tracking controller is designed to track the trajectories of a typical second-order nonlinear system with partially-unknown system dynamics and unmeasurable system states.Firstly,a neural network observer is designed based on the output information of the system.Then,based on the estimated system states,the sliding mode tracking controller is designed to track the target trajectories.(2)For nonlinear systems with constrained-input whose dynamics are partially unknown and the internal states are unmeasurable,an optimal controller is designed to stabilize the system.Firstly,the non-quadratic optimal performance index based on the system output information is defined,and the Hamilton-Jacobi-B ellman(HJB)equation of the system is derived according to the system states estimated by the neural network observer.Then,a synchronous integral reinforcement learning(SIRL)algorithm is proposed to solve the HJB equation and obtain the optimal performance index function and the optimal controller.Furthermore,the SIRL algorithm is implemented through the Actor-Critic(AC)neural networks,and the weights of the AC networks are updated simultaneously.(3)For the affine nonlinear systems with completely unknown dynamics and external disturbance,the H-infinity stabilized controller is designed.Firstly,the problem of solving Hamilton-Jacobi-Isaacs(HJI)function is transformed to solving a two-player zero-sum game.Then,the model-based policy iteration(PI)algorithm is given,and a new iteration equation is derived based on the PI algorithm,and the model-free off-policy reinforcement learning algorithm is proposed.At the same time,the optimal performance index function,optimal control input and the worst disturbance input are obtained simultaneously.Furthermore,the Actor-Critic-Disturbance(ACD)network structure is constructed during the implementation of the model-free off-policy reinforcement learning algorithm.Finally,the proposed algorithm is applied to stabilize a linear system with constrained multi-inputs,and the robust optimal controllers are designed to stabilize the complex chaotic circuit systems.