Research On Multi Vehicles Coordinated Learning Algorithm Based On Dynamic Graphs

As a popular direction in the field of artificial intelligence,autonomous driving will have an important impact on the safety of future transportation systems.Nowadays,it is difficult for single-vehicle autonomous driving technology to reach L3 level of intelligence,which makes the coordinated driving technology among networked autonomous driving vehicles attract wide attention of researchers.With the development of intelligent connected vehicle technology,autonomous driving technology is changing to network-based autonomous driving technology.Vehicle-vehicle interaction,coordinated perception,coordinated decision-making and coordinated control represent the future development direction of autonomous driving technology.Coordinated decision-making is the key to achieving human-level intelligence.It is of great significance to study it,but it also faces difficulties and challenges.Therefore,this thesis researches the coordinated decision-making problem of multi-autonomous driving vehicles under the framework of the graph model.The specific research content includes the following three aspects:First,in view of the problem of the dynamic change of the topological structure of the coordination graph and the problem of insufficient value function representational ability,this thesis proposes a multi-vehicle coordinated learning method based on the dynamic coordination graph.In the highway scene,the Driving Safety Field Theory Modeling is used to model the continuously changing topology of multi-autonomous vehicles,then the global value function is decomposed into a combination of edge-based and point-based local value functions to improve the representational ability,and the global optimal action calculated by variable elimination algorithm that realize distributed coordinated decision-making for multiple autonomous vehicles.The experimental results show that the dynamic coordination graph method we proposed can learn better driving strategies and have better performance in driving safety;Secondly,in view of the problem of poor learning efficiency of the reinforcement learning method at the early stage of training and the difficulty of accurately defining the reward function in the autonomous driving environment,this thesis proposes a graph convolutional imitation reinforcement learning method.Using excellent human driving samples as training data,by the method of generative adversarial imitation learning,the model is guided by the reward value and combined with the model-free graph convolutional reinforcement learning method.The experimental results show that our proposed method can learn driving decisions that are closer to humans,and at the same time ensure the high gradualness of the algorithm,it also greatly improves the learning efficiency of the initial algorithm training;Third,for the problem that the soft attention mechanism cannot ignore irrelevant agents to simplify the policy learning process,this thesis proposes a graph convolutional reinforcement learning method based on dynamic coordination graph model.By combining the constructed dynamic coordination graph model with the soft attention mechanism,the influence of unrelated agents is effectively reduced,the learning process is simplified,and the relationship representation between agents is refined through the attention mechanism.The experimental results show that our method can still learn safer driving strategies under the complicated road scene with the increase of the number of agents,the learning speed is faster and it has better generalization ability.