Research On Autonomous Driving Behavior Decision-making Based On Learning

Decision-making is an important topic in the research of autonomous driving,and it is a bridge between high-level perception and low-level motion planning.The research on autonomous driving behavior decision-making has important research significance and application value for improving the intelligence and safety of autonomous driving vehicles and promoting the broader development of autonomous driving vehicles.Traditional autonomous driving behavior decision-making is faced with many difficulties and challenges,such as huge state space,difficulty in design and maintenance of artificial rules,lack of rules intelligence,and so on.In this thesis,the learning methods are introduced into the behavior decision-making,and a fast and safe driving behavior decision-making algorithm is realized through the imitation learning and reinforcement learning methods for the highway and urban roads respectively.Besides,to achieve closer coordination between the decision-making layer and the motion planning layer,a reinforcement learning-based hierarchical decision-making and motion planning algorithm with rewards from motion planning cost is proposed in this thesis,and the proposed algorithm is tested in the simulation environments and a real-world vehicle platform.The main contributions are as follows:1.A decision-making method for highway driving based on imitation learning is proposed.Aiming at the "forgetting" problem in the behavior clone method,a driving behavior decision-making model based on long and short-term memory network is proposed.Aiming at the problem of low data efficiency caused by sampling with learning policy in data aggregation method,an online hybrid sampling method is proposed,which can ensure the data distribution without interference and improve learning efficiency.In this thesis,two simulation highway environments are designed,and simulation experiments are carried out in continuous action space and discrete action space respectively to verify the effectiveness of the proposed algorithm.2.A representation of urban road state considering traffic rules and a corresponding reinforcement learning method of driving behavior decision-making are proposed.The multi-mode information that needs to be considered in urban road driving is represented by the surrounding occupancy map and traffic information vector.Based on this state representation,two reinforcement learning driving behavior decision-making methods based on deep Q learning and proximal policy optimization are implemented respectively.In particular,for the intersection passing problem without traffic lights,a multi-agent intersection passing algorithm based on independent Q distribution learning is proposed.By designing different"personality" agents,the training environment of reinforcement learning is enriched and the algorithm robustness is improved.The effectiveness of the proposed algorithms is tested in multiple simulation environments.3.A reinforcement learning-based hierarchical behavior and motion planning method with rewards from motion planning cost is proposed.Aiming at the urban road driving scenes considering traffic rules,a motion planning method based on upper-level behavior is designed with generating,sampling,evaluating,and selecting of trajectories to complete the motion planning of the decision layer.The proposed hierarchical behavior and motion planning method can map the cost during the planning process as the rewards of the upper-level decision-making module,which realizes the feedback of the lower-level motion planning to the upper-level behavior decision.The performance of the algorithm is verified by ablation experiments and comparative experiments.A real vehicle experimental platform is built and the proposed hierarchical behavior and motion planning method is deployed on the real vehicle and tested on the campus road.Through the natural scene experiment and designed scene experiment,it is proved that the algorithm can directly transfer from the simulation environment to the real-world environment without fine-tuning,and can complete the navigation task safely.