Analysis Of Engine Turbocharging Control Based On Parallel Reinforcement Learning

The traditional Variable Geometry Turbocharger(VGT)control mostly uses the Proportional Integration Differentiation(PID)controller in the industrial application.However,PID control requires manual parameter adjustment,and the parameter adjustment process is complex,which requires a long parameter adjustment cycle.After parameter adjustment,the controller parameters can not change dynamically with driving conditions.Some scholars have proposed that the VGT is controlled by the Deep Deterministic Policy Gradient(DDPG)of the classical reinforcement learning algorithm.Although it can solve the problems caused by the traditional control methods,the DDPG algorithm uses single core to calculate,which has low calculation efficiency and requires large computing resources.Moreover,reinforcement learning gradually learns the control strategy through continuous interaction with the environment,which leads to the slow convergence time of the algorithm,and at the initial stage of algorithm training,the randomness of agent exploration in the environment is large,which makes it difficult for reinforcement learning algorithm to be applied in the real world.Based on the above analysis,this study proposes a parallel reinforcement learning algorithm,using parallel computing and pre training methods,which can significantly improve the computing speed of reinforcement learning,and can reduce the random exploration in the initial stage of algorithm training,and apply the classical reinforcement learning algorithm in the real world.Firstly,we use the multi-core and multi thread resources of the computer to improve the single core operation of the classical reinforcement learning algorithm,and realize the parallel calculation of the algorithm,so as to improve the calculation speed of reinforcement learning.After the completion of parallel computing,combined with the pre training method in the field of deep learning,the data obtained from PID control is used as expert data to pre train the neural network in parallel reinforcement learning algorithm,so that the random exploration in the initial training of the algorithm is reduced,and the control of VGT by the pre trained parallel reinforcement learning algorithm is completed.The research contents of this paper are as follows:(1)Establishing the average engine simulation model,study the influence of the detailed model and average model on the algorithm training,selecting the working conditions to verify the algorithm,building a co-simulation platform for the algorithm to VGT control proposed in this paper,and study the information transmission process of the algorithm between platforms.(2)The framework of parallel reinforcement learning algorithm is built to study the influence of different information in the engine as the state value of reinforcement learning on the convergence of the algorithm,and the influence of different neural network architecture on the calculation speed of the algorithm.Different number of agents are set respectively to study the cumulative reward value obtained by parallel computing,and the influence of super parameter setting on the final control performance of the algorithm to study the influence of reward value setting on the convergence of the algorithm.(3)According to the training results of parallel reinforcement learning algorithm,the control effect of PID algorithm and parallel reinforcement learning is compared,and the control performance of the algorithm before and after pre training is compared.In order to better analyze the pressure changes in the intake manifold and make the pressure visualization in the intake manifold,a three-dimensional intake manifold model is established,and the CFD simulation analysis of the model is carried out to study the pressure changes in the intake manifold under different working conditions.The results of parallel reinforcement learning algorithm show that the cumulative reward value of two agents increases by 47.42% compared with that of a single agent,and the number of convergence rounds decreases by 50%.When four agents are set,the cumulative reward value increases by 72.67% and the number of convergence rounds decreases by 81.5%.It shows that the parallel reinforcement learning algorithm can accelerate the convergence of the algorithm,and the cumulative reward value increases with the increase of the number of agents.From the pressure following effect,the integral absolute error(IAE)is reduced by 51% compared with PID control.It shows that the parallel reinforcement learning algorithm has better control performance.The results of parallel deep reinforcement learning algorithm based on pre training show that when four agents are set at the same time,the total reward value after pre training increases by 60.49% compared with the random strategy,and the number of rounds during convergence decreases by 29.73%,which indicates that the pre training method can improve the cumulative reward value and reduce the random exploration of the algorithm.From the perspective of IAE,IAE decreased by 52.4%.It shows that the random exploration after pre training is reduced,the initial performance is higher,and the convergence speed of the algorithm is faster.In this study,the parallel deep reinforcement learning algorithm is applied to control VGT,and the parallel deep reinforcement learning algorithm is applied to the field of engine supercharging to realize the dynamic adjustment of parameters and achieve the purpose of parallel computing.Firstly,it has guiding significance to the traditional control method and is of great significance to expand the application scope of classical reinforcement learning algorithm.Using the pre training method of deep learning,the reinforcement learning algorithm is applied to the real world,which has guiding significance for reinforcement learning in the current multi application simulation game.