Research On Intelligent Control Method Based On Reinforcement Learning For Hybrid Electric Vehicle

This paper takes super mild hybrid electric vehicle as object of study,for the purpose of improving fuel economy and maintaining the global balance of battery energy.Based on the reinforcement learning method,the intelligent control method of the hybrid power system is studied.From the perspective of whether the model is known,the reinforcement learning energy management strategies based on the known model（stochastic dynamic programming）and the unknown model（Q learning algorithm）are studied respectively.The main research work is introduced as followings:（1）Based on clarifying the entire vehicle structure of the super mild hybrid electric vehicle,four working modes and their corresponding energy transmission routes are analyzed.The models of the power system key components are established,which laying a foundation for subsequent vehicle control strategy research and simulation verification.（2）The basic principle of the stochastic dynamic programming method is described,and the calculation process of the strategy iteration algorithm is analyzed.Aiming at the best fuel economy,an optimization model is established which takes vehicle speed v,battery state of charge SOC,power demand P_reqas state variables,and motor torque T_m as control variable.An energy management strategy based on stochastic dynamic programming is developed.Taking the WLTP driving cycle as the simulation condition,and based on the stochastic dynamic programming control strategy,a vehicle simulation model is constructed.A comparison between the simulation results of stochastic dynamic programming method and dynamic programming method is carried out.（3）The basic principle of the Q learning algorithm is described,the connotation of the equivalent factor is analyzed,and a constant equivalent factor energy management strategy based on Q learning is formulated.Taking the WLTP driving cycle as the simulation condition,and based on the Q learning algorithm,a vehicle simulation model is constructed.A comparison between the simulation results of Q learning control strategy and stochastic dynamic programming control strategy is carried out.（4）Based on the theory of constant equivalent factor Q learning control strategy,the equivalent factor is discretized,and the initial equivalent factor optimization model is constructed.On the premise of ensuring the global balance of battery energy,and aiming at the best fuel economy of the vehicle,a method of state energy spatialization is proposed.This method converts the state energy from the time domain to the space domain,and obtains the reference energy consumption through the conversion result of the state energy spatialization.Taking the reference energy consumption as the constraint,an equivalent factor correction model based on state energy spatialization is established,and an equivalent factor offline optimization control strategy based on state energy spatialization is formulated.Combining the KL divergence rate with the state energy spatialization method,the adaptive equivalent factor control strategy based on state energy spatialization is proposed,which can adapt to various actual driving cycles.And taking a certain section of road conditions in Yubei District of Chongqing as actual driving cycle,a vehicle simulation model based on state energy spatialization is constructed.The comparison between the simulation results of the proposed strategy and the equivalent factor control strategy of Q learning is carried out.