Approximating infinite horizon discrete-time optimal control using CMAC networks

This thesis takes a novel approach to solving nonlinear optimal control problems. The method utilizes a dynamic programming approach whereby the infinite horizon discrete-time solution is encoded within two CMAC (Cerebellar Model Articulated Controller) networks both employing piecewise-linear basis functions. The forms of these neural networks are carefully chosen such that they satisfy appropriate conditions for the optimal policy and control law and are logical extensions to the linear quadratic solution for linear systems. In this manner, the approach is able to subsume the exact solution for linear problems as well as being capable of approximating the solution to nonlinear systems within an arbitrary degree of accuracy. The form of the plant for which this approach is applicable is not as restrictive as many approaches in the current literature. Included is a discussion of the choice of basis functions to be used within the CMAC networks as well as details regarding the optimization scheme used to find the parameters of the CMAC networks that best approximate the solution for a chosen set of test points within state space. The main theoretical contribution of the thesis is a rigorous treatment of the approximation of multivariate functions using a limited number of couplings. It is shown that this formulation has a polynomial growth in dimension that when applied to approximating infinite horizon optimal control has the effect of allowing the analysis of higher order systems than previously possible—an alleviation to the so-called curse of dimensionality.