Learning And Optimization Of Control Systems: Markov Performance Potential Theory And Approaches

The thesis considers the learning and optimization problem of dynamic control systems,by using the performance potential theory and approaches.Based on the core concept of potential,lots of research directions and results on learning and optimization can be unified.New theory and algorithms can be developed from the viewpoint of potential.The traditional approaches of optimal control problems can only handle special systems.For general cases,there is no simple way to address them.We apply the theory in the field of learning and optimization to the optimal control problems,and acquire some important results that can not be obtained by traditional approaches.Firstly,we extend the potential theory to continuous state space,to build a connection between dynamic systems and Markov systems.Secondly,we derive the performance potential of dynamic control systems.After having the core concept of potential, the approaches on learning and optimization,e.g.policy iteration and reinforcement learning,can be applied successfully to control problems to find the optimal control policy.The potential theory has two advantages:it retrieves structure information for optimization,and it is easy to design on-line learning algorithms.The thesis considers three classes of problems:two-level control problem of jump linear quadratic(JLQ) model,event-based control problem and constrained control problem.We formulate each class with the Markv model,and construct the equivalent Markov decision process to optimize system performance.We propose the potential of high-level modes for JLQ system,and solve the two-level control problem.With time aggregation,we formulate the optimal control problem of Lebesgue sampling system for the first time,and solve it with both analytical and sample-path-based algorithms. With performance gradient,we provide an on-line learning algorithm for the constrained control problem.