Research On Simulation Method Of Crowd Evacuation Based On Experience Sharing And Reinforcement Learning

At present,China is developing rapidly,the phenomenon of crowding has become very common,and the number of emergencies is increasing.Especially in public places with many personnel,such as schools and train stations,accidents are frequent.At present,whether the crowd can be evacuated quickly and quickly is a major safety issue in many public places.Some of the previous crowd evacuation exercises are not only ineffective,but also generally costly.It is not possible to effectively simulate random situations in accidental events.By using computer technology,it is now possible to simulate crowd evacuation,and it is more operable and consumes less cost.Crowd simulation technology has also attracted widespread attention in recent years.Through in-depth research and analysis of crowd evacuation movements,the use of certain evacuation scene modeling and evacuation simulation to design the best evacuation plan can better help people escape safely and is of great significance to humans.There are many evacuation simulation methods that can achieve group movement.Although there are many existing methods to plan the path through reinforcement learning,these methods are still insufficient,the calculation is generally large,and the process of aggregation and evacuation is relatively slow.In view of the above problems,this paper further improves the Multi-Agent Reinforcement learning algorithm,pays more attention to the communication and sharing between agents,proposes the theoretical concept of experience sharing,and applies the algorithm to the crowd evacuation.This method uses two-level control mechanism,the upper leader uses the decision-making process based on empirical knowledge reinforcement learning algorithm to select the path,and the lower crowd evacuation is mainly guided by the improved social force model.In this paper,the two improved methods are combined to improve the shortcomings of the two evacuation simulation methods when they are used alone.The macro evacuation method mainly uses the Multi-Agent Reinforcement Learning Algorithm to make the overall path planning,and then the intersection of the pedestrian tracks obtained from the real video is used as the state space of reinforcement learning,and the crowd is grouped and the leader is selected.In the reinforcement learning algorithm,a bulletin board is added to store the experience knowledge of the learning process,and the navigation agent transmits information between the leader and the bulletin board.Micro evacuation method can be said to be used to guide the movement of the crowd,which is an improved social force model.In this paper,the improved reinforcement learning algorithm and the improved social force model are effectively used together,and the actual application in the crowd evacuation simulation experiment is carried out,which successfully improves the evacuation efficiency.Finally,based on the experimental platform of the project team,the simulation platform is rebuilt.The platform is based on the algorithm proposed in this paper,and the algorithm is fully tested,which is proved to be effective.The work of this paper can be divided into three parts,which will be briefly described as follows:1.In view of the deficiency of considering the relationship between individuals in a group in social force model,an improved social force model based on aggregate force is proposed.The social force model does not consider the relationship between groups,and considers that pedestrians are moving alone,while in real life,most groups are moving in small groups.Therefore,this paper adopts a two-level mechanism model,the lower level movement uses the social force model,and we add an aggregation force into the force formula.Under the effect of cohesion,the pedestrians in the same group will attract each other,and the pedestrians in close relationship will gather together,making the evacuation more realistic.2.In this paper,an improved reinforcement learning algorithm based on experience sharing is proposed.Experience sharing is added to the original algorithm,and the multi-agent system is successfully combined.Generally speaking,the training time of reinforcement learning algorithm is relatively long,there is no communication between agents,and the learning efficiency is low.Therefore,in this paper,firstly,by using the track video tools,we extract the intersection of the travelers' tracks from the real video as the evacuation navigation point,which is the state space of reinforcement learning,reducing the number of state space.Add bulletin board to let agents share experience knowledge and communicate with each other,so that blind trial and error of reinforcement learning can be converted into path finding and selection,thus reducing the process of frequent trial and error of reinforcement learning,and quickly providing reliable evacuation path for people with evacuation.3.Combining the improved reinforcement learning algorithm with the improved social force model,a new crowd evacuation simulation system is proposed.The system is developed based on Microsoft Visual Studio 2013 and MFC,and applied to the photorealistic rendering platform developed based on XNA Game Studio 2013.It implements functions such as scene modeling,global path planning,group motion simulation,and motion status display.Taking multiple scenes as examples,evacuation simulation research and analysis are conducted,and the simulation results are displayed using the rendering output of the realistic rendering platform.Experimental results show that the simulation method proposed in this paper can perform simulation more efficiently,and it has certain reference value for crowd evacuation in real scenes,proving that the method in this paper effectively improves the evacuation efficiency.