Research On Human-like Autonomous Driving Behavior Decision-making Based On Deep Reinforcement Learning

Autonomous driving technology can improve driving safety and travel efficiency,which is an important direction for the future development of cars.As an important part of autonomous driving technology,decision-making module can select reasonable actions to complete the driving task according to the information of the perception layer,which is the core of autonomous driving technology.Aiming at the problem that the current autonomous driving decision-making is not human-like enough and less consideration is given to the driver's style,a decision-making method combining driving style is proposed based on deep reinforcement learning,which aims to meet the decision-making needs of drivers with different styles and make the decision-making more human-like.The specific research contents are as follows:(1)In view of the need to identify driving styles for autonomous driving human-like decision-making,driving styles are analyzed based on objective driving data and subjective questionnaires,and a driving style classification model is proposed.Firstly,a driving simulator is built to collect the objective driving data of the experimenters,the driving styles of the experimenters are divided into radical,general and conservative by K-means algorithm.Secondly,based on the Driver Behavior Questionnaire(DBQ)and the Multi-dimensional Driving Style Inventory(MDSI),a subjective driving style questionnaire is designed for the experimenters to fill in,after testing the reliability and validity of the questionnaire,the experimenters are divided into three categories by principal component analysis and K-means clustering,and the driving style of each category is marked.Thirdly,the results of objective driving data analysis and subjective questionnaire analysis are compared to verify the accuracy of classification.Finally,a driving style classification model is established by using artificial neural network,the experimental data collected by the driving simulator are divided into the training set and the validation set,the driving style classification model is trained through the training set,and the validation set is used to verify the accuracy of the model in recognizing driving style.(2)According to the driver's requirements for driving safety,comfort and driving efficiency,decision-making models are established based on two deep reinforcement learning algorithms: Deep Q Network(DQN)and Advantage Actor Critical(A2C).Firstly,SUMO(Simulation of Urban MObility)simulation environment is built for model training and verification.Secondly,the decision-making models are designed and trained based on DQN and A2 C algorithm,the rewards of the models consider three aspects: safety,comfort and driving efficiency.Finally,the models are tested in SUMO simulation environment,the effects of DQN and A2 C decision-making models are compared in the dimensions of reward value,collision rate,driving speed,driving distance,lane changing times and overtaking times.The results show that the vehicle using DQN decision-making model has higher safety,driving efficiency and better comprehensive performance.(3)Aiming at the problem that the current autonomous driving decision-making is not human-like enough,a human-like decision-making model combined with driving style is proposed based on the DQN decision-making model with better performance.Firstly,according to the objective driving data of three driving styles,the rewards of DQN decision-making models with different styles are designed,and the weights of safety,comfort and driving efficiency in the rewards are determined.Secondly,the DQN decision-making models of the corresponding style are trained based on the rewards of the three driving styles.Finally,the effects of different styles of DQN decision-making models are tested in SUMO simulation test environment,the results show that the vehicle with radical decision-making model has the lowest comfort and the highest driving efficiency,and the vehicle with conservative decision-making model has the highest comfort and the lowest driving efficiency,it is verified that the decision-making models of different styles can meet the driving needs of drivers with different styles,which reflects the human-like decision-making.