Research On Crossing Decision-making For Autonomous Vehicle At Intersection Based On Reinforcement Learning

As a weapon to improve urban road traffic efficiency and reduce road safety accidents,unmanned vehicles have been greatly developed in recent years,and many research institutes and universities at home and abroad have been still increasing research and development efforts.However,in order to achieve unmanned vehicles driving in a mixed real-road environment,the "control brain" of the unmanned vehicle must have the ability to learn and adapt like a human driver.The traditional rules-based vehicle intelligent driving system is only suitable for specific driving scenarios and cannot achieve self-adaptability and robustness of the automatic driving decision system.In particular,hotspots such as city intersections,where collisions occur due to a large number of vehicles,have very high requirements for making real-time correct decisions for unmanned vehicles.In this study,aiming at the problem of the cross-passage of unmanned vehicles at urban intersections,considering the factors such as the safety and efficiency of the crossing process,a method based on reinforcement learning algorithm to find the optimal crossing decision-making was proposed.For urban intersections with high traffic density and multiple conflict hotspots,this study collected a large number of empirical trajectory data of real drivers for left-turn traffic at intersections in cities,and preprocessed the data with Exponentially Weighted Moving Average method,based on dynamic clustering to obtain the decision-making interest zone for crossings at the intersection and realized dimensionality reduction of state space.Using Prescan and Matlab/Simulink to build a co-simulation platform,taking into account factors such as safety,efficiency,and comfort of passing through urban intersection,and environmental information such as relative speed and relative distance is input as the state variable of the decision-making algorithm and outputs are horizontal and vertical speed of unmanned vehicles,based on the Neural Q Learning(NQL)algorithm to establish unmanned vehicles crossing decision-making algorithm model at urban intersections,this study enabled the unmanned vehicles to cross the intersection safely and efficiently.Finally,this study designed a comparative experiment and verification process for Q-Learning and NQL algorithms in a virtual simulation scenario.The results showed that the NQL algorithm performs significantly better than the QLearning algorithm in the decision-making of continuous state space and motion space.The training sample data and training time required for the convergence of the BP neural network weight in the NQL algorithm are shorter,and the residual rate variation of the optimal action volume after convergence is controlled within 2%,but the Q-Learning algorithm requires significantly more sample data and time for convergence,and the success rate of completing the entire cross-traversal behavior is only 0.6.In the final verification of the experiment,the speed and acceleration decisions made by the NQL algorithm in both the horizontal and vertical directions are more in line with the driving rules of the human experience driver,which is safer and more efficient than Q-Learning to realize the crossing behavior of unmanned vehicles at urban intersections.