Energy Management Strategies For Hybrid Electric Vehicles Based On Real Driving Big Data And Reinforcement Learning

The energy management strategy(EMS)of hybrid electric vehicles(HEVs)has an important impact on their power demand,economy and emission performance.This paper addresses the problem that the reinforcement learning(RL)-based EMS cannot be optimally controlled online in real time due to the lack of adaptation to unfamiliar scenarios and the lack of accuracy in vehicle system modeling by combining the cyclic synthesis through real driving big data and improvement of the deep reinforcement learning algorithm adaptation to improve the learning,optimization and generalization capabilities of the EMS.The main contents are as follows:A simulation platform for optimization and validation of RL-based EMS is constructed using Python.By studying the Bellman equation form of the stochastic policy gradient algorithm and the iterative solution method,a deep reinforcement learning(DRL)algorithm combining maximum entropy theory and auto-tune soft update is proposed,which improves the exploration ability and optimality of the policy by converting the action from deterministic output to sampling over a probability distribution.The ability of the policy application to unknown environments is improved by adding a regularization process to the iterative optimization.Essentially,the algorithm improves the learning ability of the deterministic part of the transition probability and the accuracy of the estimation of the stochastic part.Based on the realistic driving scenarios of light-duty vehicles and heavy-duty vehicles,a representative driving cycle is constructed using the micro-trip(MT)method combined with the realistic driving big data of urban commuting scenarios to improve the characterization ability of the driving cycle on the speed change trend of real driving scenarios; a representative high-dimensional driving cycle is constructed using the Markov Chain-Monte Carlo(MCMC)method combined with the realistic driving big data of logistics transportation scenarios to improve the characterization ability of the driving cycle on the speed,acceleration,mass and slope change trends of real driving scenarios and their synergistic change trends.To address the problem of complex power-spliting HEV systems which are more likely to output abnormal control signals,a safety RL algorithm combining exterior point method and rule-based restraint system(RBRS)is proposed,and solve the optimization problem of energy management strategy with constraints,which improves the learning ability of the agent on the state transfer probability of the HEV system,especially the exploration ability of the system boundary model.The optimized EMS is applied to the realistic driving big data of urban commuting scenario,and the ability of the algorithm for real-time online optimal control is verified.For the hybrid action space control problem of parallel heavy-duty HEV,which contains both discrete action space such as gear shifting and continuous action space such as power distribution,a DRL algorithm for hybrid action space combining ordinal regression and representation is proposed to solve the optimization problem of gear shifting strategy in the parallel configuration and improve the learning ability of the agent on the state transfer probability of the HEV system,especially the exploration ability in the hybrid action space state domain.The optimized EMS is applied to the realistic driving big data of logistics transportation scenarios to verify the ability of the algorithm for real-time online optimal control.