Research On Vehicle Driving Strategy Optimization Based On Deep Reinforcement Learning In Virtual Environment

Intelligent driving is a major opportunity for the change in the automotive industry under the Internet wave,and also an indispensable part of the transition from the Internet era to the artificial intelligence era,in which the driving strategy of the vehicle is the core of intelligent driving.At present,there are some problems in generating vehicle driving strategies based on deep reinforcement learning,such as the high cost of sample data acquisition and low training efficiency.To solve the above problems,this thesis studies how to use the vehicle motion simulation method to generate highquality driving experience data.In addition,it also studies how to improve the training efficiency of deep reinforcement learning when generating vehicle driving strategies based on the driving experience data.The main work of this thesis is as follows:(1)Aiming at the difficulty of data acquisition and the high cost,a method of vehicle motion simulation in the virtual environment is proposed,which makes full use of the structured information and motion constraints in the environment.In the simulation process,the reference trajectory of the vehicle is generated based on the double constraints of transverse curvature and longitudinal motion,and a cascaded transverse and longitudinal separation control strategy is used to control the vehicle to track the trajectory.The Model Predictive Controller is responsible for the steering angle of the vehicle,and the Proportional-Integral-Derivative controller is responsible for controlling the gas and brake of the vehicle.The results show that the driving experience data generated based on the vehicle motion simulation method has reached the expert level.(2)Aiming at the problems of high computational complexity and low training efficiency caused by high-dimensional image input,a model-based deep reinforcement learning method is proposed to enable agents to explore vehicle driving strategies in parallel in low-dimensional state space.Firstly,based on the expert driving experience data,this method learns three models to characterize the driving environment mechanism.They are the Variational Auto-Encoders that compresses the highdimensional observation data into the low-dimensional abstract vector,the Recurrent Neural Network that predicts how the environmental state changes after driving behavior,and the environment-behavior mapping model.Then,the weight of the environment behavior mapping model is used as the initial parent of the Covariance Matrix Adaptation Evolution Strategy,and multiple agents are created on the multicore CPU processor to evaluate the evolved offspring,so as to realize the parallel exploration of vehicle driving strategy.The results show that when the driving strategy is generated based on this method,the training efficiency is improved and the driving strategy is more stable.Taking the CarRacing-v0 game scene as the simulation driving platform,the urban traffic environment and the ship surface support operation environment are constructed respectively,and the generated vehicle driving strategy is applied.The results show that the driving strategy can shorten the driving time,improve the driving efficiency,and ensure the smooth passage of vehicles in the face of different types of intersections.