Simulation Of Automatic Driving Strategy Based On Deep Reinforcement Learning

With continuous development of the global economy and urbanization process,the global car ownership and road mileage gradually increase.The consequent air pollution,traffic accidents and other environment issues increase as well.Autonomous driving technologies are the promising solutions to these problems,with the help of vehicle-to-vehicle network,intelligent driving and other intelligent car technologies.Institute of Electrical and Electronics Engineers(IEEE)predicts that the proportion of autonomous vehicles will increase up to 75% by 2040.In this paper,we propose an automatic control strategy learning method for self-driving vehicles based on deep reinforcement learning.The strategy involves pre-training with experiences from human racing car players and then training the Q-learning model with filtered experience replay(DQFE)through reinforcement learning.Since the deep Q-learning with filtered experiences method requires a long time on training,we improve the method by clustering the representative states and re-sampling the dataset from each cluster.Except of data reduction,the samples are therefore more independently and identically distributed.Therefore,the proposed Q-learning with filtered experience replay and clustering method(DQFE-C)reaches convergence rapidly in solving the object optimization problem.Experimental results demonstrate that the proposed DQFE-C model could reduce the time consumption of training by 92%,and the control stability increases by about 34%,compared with the existing neural fitted Q-iteration(NFQ)algorithm in 300 episodes.In addition,procedure on resampling after clustering could increases the average travel distance by 73.4%,compared with the Q-learning algorithm with filtered experience replay that is based on the testing track which slightly more complicated than training set.