Research On Routing Problem Of Unmanned Vehicle Distribution Based On Deep Reinforcement Learning

With the development of autonomous driving and artificial intelligence technology,unmanned logistics fleets have played an increasingly important role in the distribution of goods in urban areas.In view of the poor timeliness of traditional algorithms in solving the unmanned logistics fleet distribution in the end of the city,they often fall into sub-optimal solutions,and the expansion of the distribution scale increases the time cost index and other urgent problems to be solved.This paper proposes an improved attention-based mechanism And the method of deep reinforcement learning algorithm,and apply it to the distribution path planning problem of unmanned logistics fleet with time window and the distribution path planning problem of unmanned logistics fleet with time window considering regional congestion.This research is oriented to the problem of urban "last mile" unmanned logistics fleet distribution route planning.The cost of unmanned logistics fleet distribution routes with time windows and unmanned logistics fleet distribution routes with time windows considering regional congestion are optimized.The goal of minimizing the quantity cost is to explore the solution to the urban area logistics fleet distribution problem from the direction of artificial intelligence.By complying with the soft time window constraints and the capacity constraints of distribution vehicles,the random needs of distribution customers are met,and the distribution route planning of the unmanned logistics fleet is completed.The model represents a parameterized strategy and a parameterized value evaluation network.Reinforcement learning algorithms are used to train and optimize model parameters based on sequence rewards through round updates.The main work of this paper is as follows:(1)An improved attention mechanism model based on deep reinforcement learning algorithm is proposed.The model is based on an end-to-end idea.One end is to input the logistics fleet route problem with time window into the trained model,and the other end can quickly and efficiently give the entire fleet route scheduling.By designing a deep neural network model,building a reinforcement learning state information fusion module,an attention mechanism module,and a recurrent neural network module as a strategy network,building a value network,designing a reward function,a state transition function,and a shielding function,building a reinforcement learning algorithm and applying it Algorithm training model.(2)The built-up attention mechanism model based on deep reinforcement learning algorithms is applied to the route planning problem of unmanned logistics fleet distribution in the end of the city,focusing on the research of the unmanned delivery fleet with soft time window in the end of the city.Distribution route problem.In order to improve the speed of model convergence,solution efficiency,and solution quality,this paper continuously adjusts and improves the reward function,adding corresponding penalties to the reward function;improving the customer node shielding scheme;optimizing the reinforcement learning state transfer function and other three main aspects to optimize and improve the model.(3)On the basis of the above research,the regional congestion factor is added to the unmanned delivery fleet distribution route problem with time window,and the attention mechanism model based on deep reinforcement learning algorithm is applied to the unmanned logistics fleet delivery route considering regional congestion Planning issues.Through further modification and improvement of the model,the information processing of the congested area information(congestion radius,congestion center,congestion intensity)of the model is increased,and it is taken as the state consideration factor of deep reinforcement learning.Numerical experiments show that the model can quickly solve the vehicle routing problem with time windows for different customer node sizes,especially when the customer node distribution scale expands,it can efficiently give a good fleet planning route;the model is used in the solution of small-scale problems The greedy decoding strategy is not much different from the genetic algorithm's solution,but the model in this paper performs relatively well when the customer scale expands,and it is better than the genetic algorithm in terms of path cost and the cost of the number of cars;by comparing the two decoding strategies of customer nodes,The analysis found that the customer node in the training phase adopts a random decoding strategy,which makes the model try to explore more solution space and effectively avoid falling into local optimality.The greedy decoding method is adopted in the test phase,which can effectively improve the total path cost of the model in the fleet,the size of the fleet,and the time efficiency.And other aspects.In addition,the improved model based on the deep reinforcement learning algorithm can quickly and efficiently deal with the problem of unmanned fleet distribution route planning in urban congestion.