The Research Of Cooperative Area Search And Rescue Technology For Multi-UAVs Based On Reinforcement Learning

In recent years,factory fires,forest fires and home fires have been frequently happened,causing huge losses to people’s lives and property.As mentioned above,factory fires have the characteristics of large number of victims,concentrated storage of flammable and explosive materials,complex structure of factory buildings,etc.It is difficult for firefighters to carry out rescue accurately and quickly in the first time,which makes the disaster difficult to be controlled timely,wastes the precious rescue time,and increases the risk of firefighters.The Unmanned Aerial Vehicle(UAV)group with low cost,flexible deployment and strong mission capability can be effectively applied to the complex fire rescue environments with uncertainties and strong disturbances,and can cooperate to complete tasks such as collecting fire environment information,searching for trapped people,guiding people to escape,and resisting local fires.In this paper,UAV is taken as the basic unit of factory fire rescue,and the multi-UAVs cooperative search and rescue technology is studied.The main works are as follows:(1)In order to solve the task assignment problem of multi-UAVs under the limitation of rescue capability and the cooperative reassignment problem of emergency tasks,the fire rescue mission with multiple constraints such as time sequence,task priority and types of consumed resources under the fire scene is studied.An improved particle swarm optimization algorithm for multi-UAVs rescue task assignment is proposed.In the iterative process of the algorithm,the inertia weight of each generation is trained by reinforcement learning method.A partial replanning strategy based on the task priority is proposed,and the emergent tasks are assigned by the top-level UAV,and the executive UAVs will bid according to the maximum rescue income of the cluster on the premise of considering the remaining rescue capacity.The simulation results show that the proposed method can meet both the solution precision and convergence speed of particle swarm optimization algorithm,and can realize the reassignment of emergent tasks.(2)Aiming at the problem of cooperative search for dynamic targets in the unknown disaster environment by multi-UAVs,a cooperative search algorithm based on reinforcement learning is proposed.The method designs the reward and punishment function according to the efficiency function.The target probability map and uncertainty map are mixed to describe the unknown environment,and the "territorial awareness information map" is introduced to coordinate the cooperation among multiple UAVs.The constructed extended search map is updated online based on the search performance of multi-UAVs.The simulation results show that the algorithm is effective,and the cooperative dynamic targets search of multi-UAVs is achieved through comparative analysis,which is more effective than the original search method to realize the search of dynamic targets.(3)Aiming at the problem of path planning for multi-UAVs to guide survivors to escape from a three-dimensional environment,a three-dimensional fire environment model is established by combining the information of obstacles,survivors and combustion reaction equations of fuel beds,and Long-short Term Memory neural network is used to predict the spread of fire.A multi-objectives efficiency function is designed for the fire rescue,including the benefit of shortest path,the benefit of path safety,the cost of flight path adjustment,and the cost of collision avoidance.The Q table is trained offline by collision avoidance training and artificial potential field method.The simulation results show that the Long-short Term Memory neural network can predict the fire situation within the error range,and the improved reinforcement learning method can plan the optimal escape path for the survivors under the premise of safety.