Research On Dynamic Vehicle Sharing Scheduling Problem Based On Multi-agent

Urban intelligent transportation system is moving in the direction of on-demand travel.Unmanned driving is the future of urban transportation,and car-sharing may be a necessary stage on its development path.As the key service provided by car-sharing platform,punctuality and rationality of vehicle scheduling are guarantee of satisfaction of both users and drivers,and they are also key factors to platform's revenue.Scheduling of carsharing mainly refers to scheduling of empty vehicles in the transportation network,and planning of their matching transportation tasks.Based on analysis of actual situation,this article divides dynamic scheduling problem of car-sharing into two core sub-problems: empty vehicle scheduling problem and task assignment problem according to scheduling process and scheduling methods.Vehicle resources and travel demands in urban area have wide and random distribution.Car-sharing platforms are required to schedule empty vehicles to meet the demand of users with high requirements on commute time.The dynamic and complexity of this problem make research on transportation system under car-sharing mode more difficult,and at the same time,put forward higher requirements on solution to dynamic vehicle scheduling problem.However,normal theoretical modelling usually cannot illustrate complex system precisely.Methods taken are often compromise between timeliness of response and global optimality.However,multi-agent systems can transform real and complex systems into more natural expressions by decomposing and combining the components of the system,and have high system description ability.In recent years,multi-agent reinforcement learning has received increasing attention.Some breakthroughs in this field have brought new solutions to dynamic vehicle scheduling problem.Thus,this article models and solves problems from the perspective of multi-agent reinforcement learning.For empty vehicle scheduling problem,in order to fix the imbalance of supply and demand caused by the sharp increase in travel demand and insufficient vehicle resources and reduce waste of resources in transportation network,this article establishes a time-driven empty vehicles scheduling model based on stochastic game,which uses parameter-sharing network structure.And this article proposes scheduling algorithm based on multi-agent reinforcement learning.More precisely,vehicles are used as decision-making agents,and stages of decision-making are divided by discrete time periods.Then,extended Double Deep Q-learning algorithm is introduced and state-action value function are parameterized by neural network.By collecting experiences of agents interacting with environment,network parameters could be updated and optimal unoccupied vehicle scheduling plan could be obtained through learning.For task assignment problem,to meet the requirement of real-time response to travel needs of users,a stochastic-game-based event-driven task assignment model is built based on complementation to unoccupied vehicle scheduling model.It models nodes in transportation network as agents,vehicles at node as agents' resources.Random dynamic transportation tasks will trigger the corresponding nodes to make decisions.In order to obtain the task acceptance strategy for each agent,distributed network structure is used,and parameterize strategies of each agent based on extended ActorCritic algorithm.This framework consists of several actor networks and one centralized critic network.In training process,agents update parameters of actor networks and critic network based on experiences of interacting with environment and state value generated by critic network,and achieve ideal synergy.In testing process,agents are able to provide online decision only based on their state.Simulation and comparison experiments show that the model and algorithm proposed in this article for the dynamic scheduling problem of car-sharing can significantly improve the acceptance rate of tasks and the total revenue of transportation system,and can provide a reference for practical engineering applications.