Application Research Of Behavior Tree In Self-driving Behavior Planning Strategy

Self-driving technology,which is dedicated to solving traffic congestion and reducing traffic accidents,has been developing rapidly in recent years.One of the key open issues is to choose behavior planning strategies for effective driving behavior based on surrounding traffic conditions.A large number of current technical implementations use the finite state machine as the overall strategy model across scenes.However,there is a strong coupling among the internal modules of the finite state machine,it is not convenient for state updating or maintaining.Also,it requires expertise knowledge to design manually.On the other hand,the latest research methods of specific scenarios often use a single reinforcement learning agent to train with the entire scene,which has the problems of high state space dimension,high training difficulty,and no security guarantee caused by the neural network structure.This thesis proposes to use a more modular and more scalable behavior tree model instead of the finite state machine as the primary model of behavior planning strategy.The structure and subnode design of behavior tree is combined with genetic programming and reinforcement learning to meet the needs of different traffic scenarios.The main contents of this article are: 1)constructing 3 different traffic scenarios including pedestrians and other obstacle vehicles in the CARLA simulator: crossroads with signal lights,multi-lane straight roads and roundabout sections;Implementing the basic condition and action nodes to be used in behavior tree;2)narrowing the search space of genetic programming by normalizing the structure of the behavior tree,designing and implementing the genetic programming algorithm that combines dropout genetic operations and hash sets with fitness evaluation through the CARLA simulator;3)designing and training the behavior tree strategies with added Deep Q-Learning(DQN)nodes to solve the timing problem of importing into the roundabout.Finally,the sub-strategies in the sub-scenarios are integrated to form a behavior tree model of a complete behavior planning strategy.According to the experimental results of various scenarios in the CARLA simulator,this thesis concludes that the use of behavior trees to construct self-driving behavior planning strategies can be effectively combined with genetic programming and reinforcement learning,to reduce manual design costs and the difficulty of reinforcement learning training with enhancing the safety of reinforcement learning agents.