Study Of A Number Of Problems Of Finite-Horizon Adp Algorithm

Adaptive dynamic programming (ADP) is an effective method to solve the optimal control of nonlinear systems. For the finite horizon optimal control of nonlinear systems, a new finite-horizon ADP algorithm is proposed in recent years. However, the present finite-horizon ADP algorithm needs the one step admissible control sequence, and BP neural network (NN) is adopted to implement it. Taking the shortcomings of the algorithm and BP NN into count, the paper researches how to improve finite-horizon ADP, implement the improved algorithms by echo state network (ESN) and apply the improved algorithms to solve the control problems of dead-zone input and tracking. The main research work and conclusion includes:1. The existing finite-horizon ADP algorithm requires the controlled system can move to0in one step, that is, knowing the one step admissible control sequence. This condition blocks the application of the algorithm. To overcome the deficiency of existing algorithm, we put forward two improvement thinking, and call them improved algorithm1and2respectively. Improved algorithm1need the initial admissible control sequence, but the length of sequence is arbitrary, so it removes the limitation of the original algorithm which demand sequence length is1. Improved algorithm2don’t need initial admissible sequence, the initial cost of it has nothing to do with admissible sequence. After giving the iterative process of both two algorithms, we prove strictly the convergence of them. To obtain finite-horizon control, we introduce an error bound ε, deduce ε-optimal control, and approximate optimality of it is proved.2. We discuss the convergence of NN implement of ADP algorithm. Taking example of improved algorithm2, we analyze the approximation error of critic network, and dual network respectively. When some assumptions are satisfied, we prove the approximate cost of NN will converge to the neighborhood of optimal value. The simulation result suggests the proposed analytical method of convergence is effective.3. For nonlinear systems with dead-zone input, we solve the finite horizon optimal control by using improved algorithm1. First, a new utility function is constructed to deal with dead-zone nonlinearity. Afterwards, in the structure diagram of improved algorithm1, a dead-zone network is introduced to identify the dead-zone. The simulation example proves improved algorithm1can deal with dead-zone effectively.4. A finite horizon optimal tracking control scheme is proposed based on improved algorithm2, and ESN is adopted to carry out the scheme. The output weights of model, action and critic ESN are adjusted by LM algorithm. The simulation experiment verifies the effectiveness of proposed tracking control scheme.