Research On Adaptive Dynamic Programming Optimal Control Strategy For Data-Driven Discrete Time-Delay Systems

In the past two decades,due to the rapid development of information science and technology,many industries have undergone tremendous improvements,includ-ing robotics,metallurgy,chemical industry,microgrid,aviation,wind power gener-ation,etc.How to use the large amount of data generated in the industrial process to achieve optimal control is one of the hot issues in the field of engineering control.With the development of big data technologies,data-driven control methods have received unprecedented attention.Therefore,it is of great value to try to combine the data-driven control with the adaptive dynamic programming algorithm.First,an optimal tracking control method based on ADP algorithm is proposed for discrete systems with unknown dynamics.Secondly,a control method based on data-driven ADP algorithm is designed to effectively solve the problem of optimal control of discrete time-delay systems.Then,a new tracking control scheme based on ADP is proposed for the tracking control problem of discrete systems with disturbances.Fi-nally,the validity of the proposed methods is verified by some numerical simulations.The main contents are listed as follows:(1)The optimal tracking control approach based on adaptive dynamic program-ming(ADP)algorithm is proposed to solve the optimal tracking control prob-lem for unknown discrete-time systems with unknown dynamics.First,the optimal tracking problem is transformed into the infinite-horizon optimal reg-ulation problem of tracking error.Second,an extended error state equation is constructed using the history data of the control input and the state of the error system.Finally,the iterative ADP algorithm of policy iteration(PI)and value iteration(VI)are introduced to solve the value function of the con-trolled system.It is shown that the proposed ADP algorithm can solve the LQR problem without requiring any knowledge of the system dynamics.(2)The optimal tracking control problem for a class of linear discrete-time system-s with multiple delays is investigated based on the iterative ADP algorithm.First,by means of coordinate transformation,the minimum multiple delays e-quivalent system of multiple delays system is deduced,and then the equivalent conditions of the two systems are derived.Second,based on the augmented minimum time-delay system,an augmented performance index function with discount factor is designed.Furthermore,the corresponding Bellman equa-tion and Q-function are presented,respectively.Finally,a model-free optimal tracking controller is designed to minimize the augmented quadratic perfor-mance index based on the ADP algorithm.(3)The data-driven ADP algorithm is proposed to deal with the optimal tracking problem for the general discrete-time systems with delays for the first time.First,a model-free tracking control method based on reinforcement Q-learning is designed based on the state of the system.Second,when the system dy-namics and internal states are all unknown,the augmented state equation is constructed based on the time-delay system and the reference system.Next,based on the history data composed of input,output and reference trajectories of the system,a,new data-based augmented state equation was constructed,which is considered as a state estimator.Then,a novel data-driven Bellman equation for the linear quadratic tracking problem with delays is deduced.Fi-nally,the data-driven ADP algorithm is designed to solve the LQT problem with delays and does not require any system dynamics and internal states.(4)An optimal control method based on data-driven ADP algorithm is proposed to solve the optimal control problem for a class of discrete-time systems in the case of multiple delays.The condition of the existence of the corresponding equivalent multiple delays system is derived according to the characteristics of time-delay system.A novel data-based state equation is developed that only composed of input and output data,which is very meaningful in practical applications.Finally,a novel data-driven ADP algorithm is designed to solve the output feedback optimal control problem of linear discrete systems with multiple delays,using only the measured system input and output data.(5)H? tracking control problem for the discrete-time systems with delays is in-vestigated via the data-driven ADP algorithm.First,an augmented discrete time-delay system is constructed based on the original system and the reference trajectory system.Second,this discrete time-delay system is transformed into a system represented by the system inpnt,output,and reference trajectory,which makes the system state completely replaced by the system's operating data.Then,according to the Bellman optimality principle,a novel data-driven Bellman equation of H? tracking control problem is deduced.Finally,a con-trol method based on data-driven ADP algorithm is proposed to solve the H?tracking control problem for discrete time-delay systems.