Iterative Control Of Fermentation Process With Approximate Dynamic Programming

Approximate dynamic programming has become an effective method for solving the optimal control problem, and a hot research direction in the field of machine learning and optimal control. Recently, researches are mainly focusing on the discrete state space problems. However, the problem of practical industrial production process is usually large scale continuous state space. The existing algorithms are not a very efficient for solving these problems and learning efficiency is not high. Therefore, our most effort is focused on how to further change the structure of approximate dynamic programming and improve the learning control performance.Firstly, according to the summarization and analysis of the previous research contents, we do research on how to build a mathematical model by using neural network. For the dry point of aviation kerosene's soft sensing problem, we have established partial least squares (PLS), radial basis function neural network (RBFN) and PLS-RBFN soft sensor models. Then, we combine three soft sensor sub-models together in the form of linear weighted by principal components regression. Thus, a hybrid soft sensing prediction model is established. Consequently, this result validates the feasibility of the neural network modeling. It also provides a new approach to obtain information model for the approximate dynamic programming algorithm.In the present stage of approximate dynamic programming research, the neural network structures are usually used as the critic network. The efficiency of such structure is not high and experience knowledge is necessary for parameters selection. On the basis of the least squares temporal differences (TD) learning algorithm, the neural network structures of critic network are replaced in dual heuristic programming (DHP) by using RLSTD(0), RLSTD(?), TD with gradient correction (TDC) algorithm and the least squares temporal difference with gradient correction (LSTDC) algorithm. Hence, the improved TDC-DHP, LSTDC-DHP are derived in this paper and are compared with RLSTD(0)-DHP, RLSTD(?)-DHP. Therefore, the approximation way of value function in the critic network is improved, the weights update process is optimized and the learning control performance is enhanced.In order to verify the validity of the improved approximate dynamic programming algorithms which are derived by us, the iterative control simulation experiments are executed for the fed-batch ethanol fermentation processes. Frameworks of RLSTD(0)-DHP, RLSTD(?)-DHP, TDC-DHP and LSTDC-DHP's algorithms are given in detail. In the iterative control simulation experiments of biological fermentation batch process, the change trends of four state variables, performance index and the feed rate trajectory are observed. Simulation experimental results of four kinds of algorithms are compared and analyzed. The improved LSTDC-DHP algorithm can not only obtain the near-optimal feed rate trajectory continuously, but also achieve maximum ethanol production in the numerical point of view. Simulation results show that the improved LSTDC-DHP algorithm can simplify the weights adjustment process effectively, improves the approximation precision of the critic network and verifies the effectiveness of LSTDC-DHP in dealing with continuous space problems.