Researches On Optimal Control Method Based On Event-Triggered Adaptive Dynamic Programming

Adaptive dynamic programming(ADP)is a combination of dynamic programming,reinforcement learning,neural networks and other control theories.It provides a new and effective method to solve the optimal control problem of nonlinear systems.Compared with dynamic programming,the advantages of ADP lie in being able to effectively solve the nonlinear Hamilton-Jacobi-Bellman(HJB)equation,to overcome the "Curse of dimensionality",applicable to the complex nonlinear system with unknown system dynamics,and so on.Hence,ADP has obtained the widespread attention of researchers.However,with the development of the communication network and the increase in the number of computing data,the traditional time-triggered ADP algorithm has been proved difficult to meet the requirements of the computational and resource utilization efficiency.Therefore,based on the event-triggered control and the ADP theory,a novel event-triggered ADP algorithm is put forward in this dissertation and the stability of the closed-loop system is guaranteed from a theoretical point of view,which provides new ideas and new implementations for the analysis and control of complex nonlinear systems.In the proposed event-triggered ADP control,the controller is updated in an aperiodic manner as well as the data transmission,which can greatly improve the efficiency of computing and communication resources.The main researches of the dissertation are summarized as follows:(1)A predictive event-triggered heuristic dynamic programming(HDP)algorithm is proposed to solve the optimal control problem for a class of unknown nonlinear continuous-time systems.Generally,in the event-triggered HDP control,the event-triggered error is defined as the difference between the current state and the last sampled state,and thus the appropriate event-triggered condition is designed.The method proposed in this dissertation takes advantage of the powerful nonlinear mapping ability of neural networks to reconstruct the system model and then estimate the state vector.On the basis of the traditional event-triggered control,a predictive event-triggered error can be calculated with the estimated future system information,and then a predictive event-triggered condition is designed.The plant can converge faster and save more computational cost while the controller is updated under the predictive event-triggered mechanism.(2)An event-triggered HDP algorithm is proposed for the optimal control problem of a class of nonlinear continuous-time systems with control constraints.Firstly,inspired by Lyshevski,a performance index function in nonquadratic form is designed to solve the optimal control problem of control constrained systems.Secondly,since it can hardly solve the HJB equation of the nonlinear control constrained systems' directly and obtain the analytical solution,by taking use of the nonlinear mapping ability and strong adaptive learning ability of the neural networks,an actor-critic structure is adopted to obtain the approximate solution of the HJB equation.Meanwhile,Lyapunov stability theory is applied to design the appropriate trigger threshold to push the HDP controller updated in an aperiodic manner under the designed event-triggered condition.So that the efficiency and the control performance of the controller can be guaranteed.Finally,a rigorous mathematical proof is given to ensure the stability of the closed-loop system.(3)An event-triggered dual heuristic programming(DHP)algorithm is presented to solve the optimal control problem for a class of complex nonlinear continuous-time systems.In the traditional DHP algorithm,the output of the critic network is defined as the co state,that is,the partial derivative of the cost function.The dimension of the costate is affected by the input dimension of the critic network,and therefore more information is included.With the increase of the input dimension of the critic network,the computional complexity of the DHP controller will increase exponentially,which leads to the fact that the application of the controller will be limited in large-scale complex systems.The difficulty of the proposed method lies in that as the complexity of the output of the critic network increase,the difficulty of designing the event-triggered condition is also increasing.In this dissertation,by using Lyapunov stability theory,the trigger threshold is designed for the DHP controller,and then an event-triggered condition for aperiodic sampling is provided.In the meantime,the stability of the system and the convergence of the neural networks are proved under the designed sampling mechanism.(4)A novel event-triggered HDP controller is proposed to solve the optimal control problem of a class of nonlinear discrete-time systems.It is assumed that the plant is with input-to-state stability(ISS)property,and thus the ISS-Lyapunov function is defined,on the basis of which the trigger threshold is designed.It is demonstrated that the plant is asymptotically stable under the designed event-triggered controller.Simulation results show that compared with the traditional time-triggered HDP,the proposed method can significantly reduce the computational cost and still ensure the control performance.(5)Load frequency control(LFC)is one of the major subjects in the power system and has been attracted extensive attention from researchers.To ensure the effectiveness and stability of LFC,the proportion-integral(PI)controller is set as the major controller.At the same time,in order to compensate for the lack of adaptive learning ability,the ADP controller is added as the supplementary controller.The aforementioned design can not only keep the system information in the preset controller,but also take advantage of the learning ability of neural networks to improve the robustness and adaptability of the algorithm under the premise of ensuring the stability.However,the design of dual controller yields a higher computational cost.In order to reduce the computational cost and the transmission burden,a novel event-triggered PI-ADP controller is proposed.In this design,the event-triggered PI controller and the event-triggered ADP controller are designed separately with different trigger thresholds,and it is proven that the aperiodic update law can guarantee the stability of the closed-loop system.Finally,the concluding remarks are included.On the basis of summing up the work of the dissertation,the further research directions of event-triggered ADP algorithm are pointed out and the prospects of future work are given.