Research On Energy Management Strategy Of Hybrid Electric Vehicle Based On Reinforcement Learning

Facing the pressure of environmental pollution and energy consumption,new energy vehicles become a new trend of automotive technology development.Hybrid Electric Vehicle(HEV),which combines the advantages of traditional fuel vehicle and pure electric vehicle,has good fuel economy without being limited by the battery power,and is a hot topic in current research.As a core part of HEV,energy management strategy(EMS),can directly affect the vehicle fuel-saving performance by regulating energy flow between engine and battery.As a new intelligent decision-making optimization algorithm,reinforcement learning(RL)provides a new opportunity for the design of EMS of HEV.However,due to the complexity of energy management problems,"curse of dimensionality" and other problems hinder the application of RL in EMS.Taking the coaxial parallel hybrid electric vehicle as the research object,this paper studies the application of reinforcement learning algorithm in energy management strategy design for variable driving cycles.The main contents are as follows:Aiming at the problem of high cost and high risk of vehicle test,this paper builds a test bench based on the powertrain of coaxial parallel hybrid electric vehicle.The PXI is selected as the lower computer controller to control the test bench in real time.The control program,communication program and real-time monitoring interface of the test bench are designed by using Matlab/Simulink and Labview.Based on the test bench,the vehicle simulation model is built by using Matlab/Simulink and Simscape,which provides the basis for strategy training and verification.The parameters of the simulation model are modified by genetic algorithm to ensure that the results of simulation training have reference significance for bench verification.Aiming at the problems of "dimension disaster" in traditional reinforcement learning and over fitting in deep Q-network(DQN)algorithm,this paper studies the application of double deep Q-networks(DDQN)algorithm in energy management problem.The strategy replaces the policy matrix with deep neural network(DNN)and combines it with Q-learning algorithm.It can achieve good fuel economy only by relying on the current information of the vehicle and can be applied to variable driving cycles.First,this paper illustrates the algorithm theory of RL and discusses the application of reinforcement learning on energy management strategy.The theory of DDQN algorithm and the framework of DDQN-based EMS are described in detail.The setting of state,action and reward which are the key elements in DDQN strategy and the design of Q-network and Q’-network which are used for action selection and target Q-value calculation are introduced in detail respectively.The algorithm flow of DDQN strategy is described in the form of pseudo-code.The effectiveness and adaptability of the strategy are verified by simulation experiments under NYCC,FTP and UDDS driving cycles.Intelligence,networking and informatization are the future development trend of automobile.In order to avoid the introduction of traffic information to increase the burden of strategy training,a DDQN-based EMS with demand torque prediction is proposed.The strategy applies BP neural network algorithm to predict the future demand torque through the vehicle speed in front,the vehicle speed and the distance between two vehicles.Then the predicted demand torque is introduced into DDQN strategy as state.The simulation results show that the introduction of predicted demand torque can reduce fuel consumption significantly and the strategy can be applied to different driving cycles.