Research On Energy Management Strategy For Plug-in Hybrid Electric Vehicle Based On Reinforcement Learning

With the pressure of environmental degradation and energy shortage,plug-in hybrid electric vehicle have become the new energy vehicle type with the most development potential and market prospect.As a key technology of PHEV,the energy management strategy directly affects the vehicle fuel economy and emission capability.In this paper,the power-split PHEV is taken as the research object to research three different energy management strategies: the energy management strategy based on stochastic dynamic programming,reinforcement learning and stochastic model predictive control.The specific research contents include:(1)The power system structure of power-split PHEV is introduced in detail,the energy flow of each power component in different working modes is analyzed.Then,the energy management control model of power-split PHEV is established,and the control variables and objective functions of the energy management optimization in this paper are determined.(2)The energy management strategy based on stochastic dynamic programming is investigated.Firstly,the principle of stochastic dynamic programming is introduced,three methods for solving stochastic dynamic programming problems are analyzed and compared.The transfer probability of the vehicle power demand is devoted to express the uncertainty of future driving cycle.The demand power of several standard driving cycles are discretized nonlinearly and modeled as a Markov Chain.The modified policy iteration method is imposed to solve the stochastic dynamic programming control problem and the offline optimal battery power sequence based on stochastic dynamic programming is obtained.Online simulation results show that the stochastic dynamic programming strategy can effectively reduce fuel consumption and it has strong adaptability to different driving cycles.(3)The energy management strategy based on reinforcement learning is investigated.Firstly,the relationship between reinforcement learning and Markov Decision Processes is analyzed,the basic principle of reinforcement learning is described,and three kinds of reinforcement learning algorithms are compared.The Q-learning algorithm with fast convergence is determined as the energy management control method.The Q-learning algorithm is employed to model the energy management problem,combined with the demand power Markov Chain model established in SDP strategy.And the method of value iteration is applied to solve the optimal control problem of Q-learning,and a offline optimal Q-learning controller is obtained.The convergence of the Q-learning algorithm in different states is analyzed and ensure the convergence of the control algorithm.Simulation results show that the Q-learning mehod can effectively improve fuel economy and achieve global sub-optimal results similar to SDP.(4)The energy management strategy based on stochastic model predictive control is proposed.Markov Chain model based on acceleration is established,and two random velocity prediction methods are proposed.The Q-learning algorithm controller is employed in the rolling optimization process to establish an online stochastic model prediction controller based on RL.The simulation results show that the controller can effectively improve the fuel consumption rate and obtain similar results to the stochastic dynamic programming method.In addition,the influence of prediction accuracy on the controller,the proposed controller follow-up effect on different state of charge reference trajectories and the calculation efficiency of different driving cycles are analyzed,the proposed strategy is proved to be able to guarantee computational efficiency and improve fuel economy...