| Lane keeping is an important part of realizing vehicle autonomous driving.Deep reinforcement learning algorithm is a mature machine self-learning algorithm.This paper studies the vehicle lane keeping algorithm based on deep reinforcement learning.Aiming at the infinity of the task space and the continuity of the action space in the actual task process,the DDPG algorithm is selected as the core algorithm in the lane keeping self-learning process,and the overfitting prevention is improved on the basis of this algorithm.Then,the design and implementation of lane self-keeping system based on multi-architecture fusion are studied,including image processing research,experience playback research and exploration research,sparse reward research,etc.(1)In terms of image processing research,in order to reduce the difficulty of self-learning of agents,image processing algorithms commonly used in deep learning are borrowed,such as Auto-Encode algorithm,transfer learning,and excellent performance in image classification and target recognition tasks.Resnet network and Densenet network.And make it in the shallow layer of the relevant neural network in the agent,thereby effectively improving the learning speed of the agent.(2)In the study of experience replay,two methods,random experience replay and priority experience replay,are used.When using the transfer learning method,due to the slow learning of the transfer network and the limitation of hardware,random experience playback like DQN is used for training.When using the Auto-Encode method,the priority experience playback method is used.(3)In terms of sparse reward research,various problems that may exist in the process of environmental feedback reward are introduced,and a set of reward calculation formulas are proposed for the sparse reward problem in the task.The success probability of the task,this paper randomly adds several groups of successful artificial cases in the process of agent learning,in order to speed up the learning process of the agent.(4)In terms of agent exploration,in view of the problem that the agent uses maximizing action value during the learning process,which leads to the problem of falling into the local optimal solution,and at the same time,for the problem that the traditional greedy method performs poorly in the long-term learning process of the agent,this paper adopts The exploration strategy of ε-greedy.The exploration strategy can make the agent maintain good exploration performance during the development process.Finally,the Carla driverless simulator is introduced,and the algorithm of this paper is verified in the simulation environment.Experiments have shown that the vehicle agent using the lane self-keeping system based on multi-architecture fusion successfully learns the control strategy of lane keeping after about 400,000 times of training,that is,using the idea of deep reinforcement learning to solve related problems in autonomous driving is It is possible,and the learning performance of the neural network is better when using the Auto-Encode method and the way of prioritizing experience playback. |
