Design And Implementation Of Virtual Simulation Platform For Multi-Agent System

Compared with single UAV system,multi-UAV system has the characteristics of high fault tolerance rate and large load.It has strong adaptability to complex task environment and has important research value.However,its large scale and high cost lead to the difficulty of physical research.In order to shorten the research cycle and improve the development efficiency,this paper builds a virtual simulation platform of multi-UAV system for urban environment,which is used to display diverse urban mission scenarios and support online training and verification of intelligent algorithms.The main research contents include:(1)Develop data management software for unified management of simulation data according to the characteristics of diversified mission scenarios of multi-UAV system.First,based on Swing framework design graphical user interface.After that,the video display module is developed to demonstrate the multi-UAV mission scenarios in the virtual urban environment.Second,according to the number of data interaction,the simulation types are divided into single interactive simulation and multiple interaction simulation.The former is represented by cluster task allocation while the latter is represented by the hunting and red-blue game mission,mainly uses reinforcement learning method to continuously train the model in "trial and error".The simulation data storage module is designed based on database and message queue interaction respectively.Finally,the simulation data supervision module is developed based on My SQL database.(2)Visual software is developed to meet the demonstration requirements of UAV cluster multi-type tasks and the requirements of intelligent algorithm for training environment.The virtual urban environment and various target scenario are built in Unity game engine by using 3ds Max modeling tool and Fantastic City Generator and other scene building tools,and the task scenarios are divided into demonstration scenario and training scenario.The demonstration scene is completely realized in the virtual environment.The C# script is used to drive the scene.Animation effects,track plug-in and other technologies are used to simulate the mission scenarios of the multiUAV system in the urban environment.The training scenario considers the data interaction with the algorithm.Developed based on the ML-Agents framework,the environment data is used to help strengthen the training of the learning algorithm and the training status is intuitively observed through visual demonstration.(3)In view of the dynamic environment of multi-UAV combat,taking the hunting mission as an example,the hunting model is designed considering the mission objectives and constraints.As a result of the multi-agent reinforcement learning network training needs,consider the behavior of the other UAVs in the virtual task environment decision,to hunting scenario and red-blue game scenario,based on the pathfinding method,the artificial potential field method,the clustering method and the proportion control method to design the strategy of the enemy,purpose is through the interaction of both sides behavior training out the reasonable network parameters,and make the execution effect of the task more close to the real task environment.