Aero-engine Intelligent Control Based On Reinforcement Learning

The aero-engine is a complex thermal mechanical system with wide flight envelope,strong nonlinearity,uncertainty and time-varying characteristics.In this paper,a turbofan engine and a micro turbojet engine are taken as the control objects to study the control algorithm based on reinforcement learning.Firstly,the theory and algorithm of reinforcement learning were studied,which lay the foundation for the design of the control algorithm.Secondly,based on the T-MATS component-level model of a turbofan engine,the dynamic link library of the model was generated by using automatic code generation technology.Thirdly,the state space and action space representation of the engine were established,the goal-based reward function and the control algorithm were designed.The algorithm consists of four deep neural networks: actor network,target actor network,critic network and target critic network.And deep deterministic policy gradient(DDPG)and batch normalization techniques were used.The simulation results showed that the control algorithm is adaptive,fast response,small overshoot,and the control accuracy meets the requirements.Aiming at the rotational speed control of micro turbojet engine(MTE),a pre-training based real-time MTE control algorithm was proposed.By pre-training the actor network,the convergence speed of the network during the training process can be accelerated,and the number of trial and error can be reduced.At the same time,by using multi-threading technology,the main control thread and the update thread run independently,and the controller parameters are updated regularly,thereby avoiding real-time problems caused by long training time of small batch samples.In the end,the effectiveness of the control algorithm was verified by the simulation experiments in the loop.