Distributed Hierarchical Control For Connected Hybrid Electric Vehicles Platooning Based On Deep Reinforcement Learning

With the rapid increase of automobiles’ conservative quantity all over the world,environmental pollution,traffic congestion,frequent traffic accidents,energy shortage,are increasingly prominent.Researches have shown that vehicle platoon control based on communications technology can improve road capacity and traffic flow,and effectively reduce traffic accidents and traffic congestion.The current intelligent connected vehicle platoon control mostly is electric vehicle platoon control,which has some problems,such as poor riding comfort,less application in hybrid electric vehicles,unable to give full play to the energy saving potential of hybrid electric vehicles,etc.One of the major sticking points in the platoon control algorithm is improving the ride comfort and fuel economy on the premise of ensuring the safety and stability of the platoon.In this thesis,a distributed hierarchical control for connected hybrid electric vehicles platooning based on deep reinforcement learning is proposed.The kinematic model of CHEVs platooning is established,and the upper model predictive control based multi-objective control law and the lower DRL based multi-system dynamic coordinated control law are designed.The major research contents of this thesis are as following:(1)Building kinematic model of CHEVs platooning.Kinematic model of CHEVs platooning includes vehicle dynamics model and distance model.First,based on the collected experimental data of power split hybrid electric vehicle power system,and build the dynamic models of each component in Python by combining theoretical and physical modeling,so as to provide controlled environment for subsequent distributed hierarchical control for connected hybrid electric vehicles platooning.After analyzing the input and output characteristics of the acceleration of CHEV and different distance models,the fixed headway is selected to establish the distance model.(2)A multi-objective longitudinal platoon control method based on MPC was proposed.The communication layer processes the vehicle information in the platoon.The robust prediction model based on state feedback error is used to eliminate the model mismatch caused by parameter error or vehicle dynamic characteristics change,and enhance the accuracy and robustness of the prediction model.After analyzing the multi-objective optimization problems of vehicle safety,comfort and economy,we establish the objective function,effectively solves the contradiction of multi-objective requirements for vehicle control,and realizes the effective combination of model predictive control and hybrid electric vehicle platoon control.The upper model predictive control based on multi-objective can achieve a good queue safety,stability and ride comfort in WLTC driving cycle.(3)A under layer dynamic coordinated control method based on deep reinforcement learning is proposed.The lower control method calculates engine and motor power allocation schemes to optimize the fuel performance based on the vehicle condition and the optimal expected acceleration for the upper control law.Using the expert knowledge composed of engine optimal working curve and battery characteristics can narrow the search scope of the algorithm,reduce the dimension of control quantity and the calculation burden,and improve the calculation speed of the algorithm.The deep Q-learning algorithm which performs well in discrete space is adopted to solve the problem of power allocation.The deep deterministic policy gradient algorithm(DDPG)which performs well in continuous pace optimization,is used to solve the problem of control quantity inaccuracy caused by discretization.The training results in NEDC driving cycle indicate that DQN-based strategy makes a better performance than that of DDPG-based strategy in training time consuming,but worse than convergence rate and fuel economy.(4)Analysis and simulation.The IM240 and HWFET driving cycles were selected to verify the feasibility of the proposed hierarchical control method for the queue of CHEVs.The experimental results show that the proposed hierarchical control system can improve the tracking ability of vehicles in the queue,and significantly improve the fuel economy and ride comfort of vehicles.But the DDPG-based CHEVs platoon hierarchical control system has a better adaptability and higher fuel economy.