A Decision-making Method For Self-driving Based On Deep Reinforcement Learning

Self-driving vehicle is a multi-disciplinary complexes of sensors,network communications,navigation and positioning,artificial intelligence,etc.Among them,navigation and positioning,path planning,behavioral decision-making,and vehicle control are critical technologies for self-driving.This thesis studies the behavior decision-making part of self-driving.With the rapid development of AI technology,it has become one of the hotspots to study self-driving technology through deep reinforcement learning algorithms to realize self-driving behavior decisions.In this thesis,the practice,improvement,and simulation verification of self-driving decision-making method based on deep reinforcement learning algorithm will be carried out under the virtual environment.First of all,it is introduced that reinforcement learning matches the state and action strategies through the trial and error learning of the agent in the environment.From the components of reinforcement learning and the learning process,several reinforcement learning principles,frameworks and network structures of reinforcement learning algorithms are introduced in detail.It introduces the combination method of the main algorithm reinforcement learning and deep learning in this thesis —— deep reinforcement learning method.Deep Deterministic Policy Gradient(DDPG),as one of the classic algorithms in deep reinforcement learning algorithm,can generate deterministic action strategies.The Soft Actor-Critic(SAC)algorithm adds the maximum entropy term on the basis of the original deep reinforcement learning objective function and shows great advantages in continuous control problems.Secondly,based on the open-source platform TORCS,this thesis uses these two algorithms to conduct self-driving simulation experiments.During the experiment,a reward function based on simulation environment information and relevant traffic rules was designed,driving stability constraints were added,and information about radar ranging sensors around the vehicle body was used to make self-driving decisions.At the same time,in the comparison experiment of the existing DDPG algorithm and SAC algorithm for self-driving simulation,the problems such as slow convergence speed of the SAC algorithm are found.Finally,an improved SAC algorithm that changes the storage method of the experience pool is proposed.In the comparative experiment,it is verified that the improved SAC algorithm with increasing and decreasing experience pool proposed in this thesis performs well in shortening training time,improving algorithm stability,and improving the generalization ability of algorithm models.This thesis also defines the error rate as a reference indicator.The error rate of the improved SAC algorithm model during testing is reduced by about 10% compared to the DDPG algorithm model.