Research And Application Of Reinforcement Learning In Autonomous Driving Research On Automatic Driving Algorithm Based On Prior Knowledge In Multiple Driving Scenarios

Autonomous driving technology is not a single point of technology,but the integration of multiple technologies.The overall technical architecture of unmanned driving can be roughly divided into three modules:algorithms,systems,and cloud platforms.With the application of intelligent driving products,the functions of the intelligent driving system's perception system and execution system are insufficient,the algorithm tests of perception algorithms,decision-making algorithms,control algorithms,etc.are insufficient,and personnel misuse,that is,the expected functional safety problems of intelligent driving,give The life and property safety risks brought by consumers or related parties have gradually become the direction of research and improvement of intelligent driving systems.In the real environment,it is impossible to train an automatic driving system vehicle on the road in order to train an algorithm model,which is extremely costly and unrealistic.Because some algorithm models such as reinforcement learning(RL)training often undergo constant trial and error to fit a state-action-reward suitable for a specified road section.Under this kind of process,the vehicle will drive aimlessly,hit the edge of the road or lose control.If the conditions are not met,the vehicle will return to the initial position and continue training until the vehicle reaches a better position on the road after a certain period of training.Complete the smooth point-to-point driving of the road section.Therefore,the driverless control algorithm module has some problems to be solved in the algorithm verification before the actual vehicle test:(1)Before the actual test verification starts,a large number of virtual simulation tests are required to ensure the feasibility and robustness of the algorithm There is no prerequisite to guarantee the accuracy and reliability of the test,and the actual vehicle will not be very efficient to the automatic driving control algorithm;(2)The automatic driving control algorithm is based on the expert system in the actual production environment.The main large and small state machines jump and distribute a fixed control amount.The unsmooth control amount will cause the frustration of the real vehicle;(3)Automatic driving control algorithm based on reinforcement learning and imitation learning(IL),is often a single-point algorithm model,which requires a lot of training time to fit the optimal control curve,resulting in low training efficiency.In order to solve the above three problems,this paper focuses on the TRPO algorithm based on the knowledge of reinforcement learning,imitation learning and potential energy field,and has made some improvements to it.The main research work and results are as follows:1.This paper builds an autonomous driving virtual simulation platform based on model-in-the-loop(MiL).The purpose is to shorten the test cycle of autonomous vehicles under the prerequisites of ensuring test accuracy and reliability;the method is to use carla simulator(carla simulator)established a virtual simulation model of autonomous driving.According to the principle of realism,use game engines,collision detection sensors and custom maps to optimize virtual simulation scenarios to construct different driving scenarios;The result is to output the control amount changes in different scenarios;The conclusion is to verify the usability of the platform and speed up the test efficiency.2.This paper compares and analyzes the key algorithms of reinforcement learning:DDPG,A3C and DQN algorithm models and imitation learning algorithm models on the basis of a virtual simulation platform built for autonomous driving.These several algorithms and data visualization performance in a simulation environment,comparative analysis,and proposed several advantages and disadvantages of this algorithm model.3.Based on the comparative analysis and research on the algorithm models of reinforcement learning and imitation learning,this paper designs a path planning algorithm based on potential energy field knowledge and imitation learning fusion,and explores whether the simulation vehicle has strong self-restraint to changes in the environment.This fusion of multi-sensor simulation car path based on reinforcement learning and imitation learning potential field of knowledge in programming algorithm,is the data packet(rosbag)collected under the robot operating system(ROS)architecture,as a priori knowledge,heuristic initialization of each state information,make reinforcement learning have a certain degree of guidance in the early stage of learning,since the lane and points(way points)as the mid-potential value in the learning,the guide agent(agent)as a gravitational bias,while improving the efficiency of the algorithm also reduces the learning time.Utilizing the feature that the environment of the carla simulator can be dynamically configured,target points are added to the environment,and each target point is configured with a gravitational potential field or a repulsion potential field.The gravitational potential field can guide the simulator to move toward it,and the repulsion field can prevent simulation vehicle approaches it.As an environmental constraint of prior knowledge,it can reduce the frequency of meaningless training of the simulation vehicle under the reinforcement learning and simulation learning fusion multi-sensor simulation vehicle path planning algorithm,which greatly increases the training efficiency and directly affects the final experimental results.