Self-learning Optimal Control Of Nonlinear Two-time-scale Systems

The nonlinear two-time-scale(TTS)systems are complex systems with fast and slow changing dynamics,which are widely used in process industry,aerospace,smart grids and other fields.Due to the high-order characteristics of the systems and the coupling between fast and slow dynamics,high-dimensional and ill-conditioned numerical problems may occur during the performance analysis and controller design.At the same time,the problems of time delays,disturbances and unmodeled dynamics commonly existed in practical systems bring great challenges to the analysis and design of nonlinear TTS systems.The existing robust control and optimal control methods are highly dependent on disturbance information or model dynamic information and lack of self-learning ability.Therefore,it is of great significance to study the self-learning optimal control methods of nonlinear TTS systems.In this paper,by combining the singular perturbation theory with inverse optimal control,sliding mode control and reinforcement learning methods,a series of self-learning optimal control algorithms are proposed,which can effectively overcome high-dimensional and ill-conditioned numerical problems.The main results of this paper are as follows:(1)For a class of nonlinear TTS neural networks with time delays,stability criterion and inverse optimal synchronization control method are proposed.First,for nonlinear TTS neural networks with multiple time-varying delays,by constructing a delay-dependent Lyapunov-Krasovskii functional related to the time scale parameter?,the sufficient conditions for the asymptotic stability of the systems are established and the estimation method of stability bound is given.Then,for nonlinear TTS neural networks with constant delays,a control Lyapunov function that depends on the time-scale parameter ? is constructed,and a state feedback synchronization optimal control method is proposed by combining the Hamilton-Jacobi-Bellman(HJB)equation with inverse optimality techniques.The ill-conditioned numerical problems can be effectively avoided.Finally,the simulation on numerical examples shows that the proposed stability criterion is less conservative and the designed synchronization controller can make two systems synchronize faster.(2)For a class of TTS systems with unknown disturbances,a disturbance suppression method based on adaptive sliding mode control is proposed.Firstly,the block diagonalization approach is introduced to decompose the full-order systems into fast and slow subsystems.Then,an equivalent input disturbance is constructed to estimate the unknown disturbances.Based on the reduced-order subsystems,a composite sliding mode surface is constructed using Lyapunov equation.Combined with the estimation of the equivalent input disturbance,the adaptive sliding mode controller is designed and the reachability condition is proved.The high-dimensional and ill-conditioned numerical problems can be avoided in the design process.Finally,the simulation on the magnetic tape control system verifies that the proposed control method can adaptively compensate the adverse effects of disturbances without knowing any prior disturbance information.(3)For a class of nonlinear TTS systems with unknown slow dynamics,a composite optimal control approach based on reinforcement learning and T-S fuzzy methods is proposed.First,using singular perturbation theory,the original optimal control problem is transformed into two reduced-order subproblems.Then,in order to solve the slow subproblem,a nonlinear coordinate transformation is introduced to deal with unknown non-standard slow utility function,and the slow controller design algorithm is proposed based on reinforcement learning.Considering the slow time-varying characteristics of the fast subsystem,T-S fuzzy fast model is established and the fast controller is designed with the approach of parallel distributed compensation.Considering the multi-source approximation errors,convergence of the slow controller design algorithm,suboptimality of the composite controller and stability of the closed-loop systems are proved.The high-dimensional and ill-conditioned numerical problems can be effectively avoided in the design process.Finally,the simulation on the numerical example and motor system illustrates that when the slow dynamics are unknown,the designed composite controller can be O(?)-close to the optimal controller and can make the closed-loop TTS systems asymptotically stable.(4)For a class of nonlinear TTS systems with completely unknown dynamics,a reduced optimal control method is proposed based on reinforcement learning.First,using singular perturbation theory,the original system is reduced to a lower-order system,by which a policy iteration method is proposed to solve the corresponding reduced HJB equation with convergence guaranteed.Then,the slow state measurements of the original system are used to reconstruct the unmeasurable reduced-order system state,and the actor-critic neural networks are used to approximate the reduced-order controller and performance index.The policy iteration algorithm is implemented under the framework of reinforcement learning and the weights of neural networks are updated by the method of weighted residuals.Considering the neural network approximation error and state reconstruction error,convergence of the iteration algorithm,suboptimality of the reduced controller and stability of the closed-loop systems are proved.The high-dimensional and ill-conditioned numerical problems can be effectively avoided in the design process.Finally,the simulation on the numerical example and inverted pendulum system shows that when the dynamics are completely unknown,the designed reduced controller can be O(?)-close to the optimal controller and can make the closed-loop TTS systems asymptotically stable.This dissertation consists of 29 figures,8 tables and 164 references.