Research On The End-to-end Strategy Learning Methods In Automatic Driving

Intelligent transportation is an inevitable trend of social development and technological development.The research and popularization of intelligent driving vehicles can greatly alleviate a series of social problems caused by the sharp increase in the number of traditional motor vehicles,such as pollution caused by vehicle exhaust,road expansion brings about the shortage of land resources and traffic accidents.In the field of autonomous driving research,reinforcement learning is favored by autonomous driving researchers because it is good at solving sequential decision-making problems.This topic focuses on the optimization of deep reinforcement learning algorithms,and relies on intelligent driving tasks for verification and comparative analysis.Aiming at the problem that deep reinforcement learning needs a lot of trial and error,which leads to low learning efficiency,this thesis uses two different ideas to combine Imitation Learning(IL)and Deep Deterministic Policy Gradient(DDPG),thus proposing two new end-to-end learning methods for human-like driving policy learning of intelligent driving systems.First,a Deep Deterministic Policy Gradient algorithm framework based on Imitation Learning(DDPG-IL)is proposed.The algorithm takes radar and other sensor data as input and directly outputs vehicle control commands.The framework is divided into two parts: the imitation learning part,which first trains the imitation learning network by collecting a small amount of label data,and then uses the trained imitation learning network to generate demonstration data;the deep reinforcement learning part,the deep deterministic policy gradient network is initialized using the data obtained by imitation learning.When learning driving strategies online,the learning efficiency of the algorithm is further improved by building a dual experience cache pool to dynamically allocate the learning ratio of demo data and exploration data.Second,a vision-based Deep Imitation Reinforcement Learning(DIRL)framework is proposed.The algorithm solves the direct conversion of driving images from the first-person perspective into driving instructions of the vehicle.The framework divides autonomous driving decision-making into two parts: perception model and control model.The perception model uses the IL network as the encoder to process the input driving image into a low-dimensional feature vector.The control model is constructed using the DDPG algorithm and receives the feature vector from the perception model to output vehicle control instructions.During the training process,the IL network is trained by collecting a small amount of label data,and the well-trained IL network is used to initialize the Actor network of the DDPG to improve the efficiency of exploration.In addition,by defining the driving reward function for human-like driving,the safety and stability of autonomous vehicles on curves are improved.In addition,the simulation experiment and result analysis of the two schemes were carried out by using the simulation experiment platform of the simulator(The Open Racing Car Simulator,TORCS).The label data used to train the IL network in the experiment comes from manually operating the racing car to collect driving data.Through experimental comparison with traditional algorithms and other improved algorithms,the simulation results show that the DDPG-IL algorithm proposed in Option I has an increase in the rate of learning driving strategies by about 20%,and the DIRL algorithm proposed in Option II has an average learning efficiency improvement of 30%,and the driving safety in corners is improved significantly.