Research On Automatic Parking Lot Scheduling Method Based On Reinforcement Learning

With the rapid development of economy and society,the occupancy of automobiles is increasing.The traditional parking lot has the shortcomings of low parking efficiency and low utilization of parking area.At present,a new type of automated parking lot has emerged in which the parking robots carry vehicles between entrance and parking space.The optimization strategy of parking lot dispatching needs to arrange suitable parking spaces according to the spatial distribution of parking spaces and the information of parking vehicles,and reduces the energy consumption caused by the long-term operation of the automatic parking lot.It is of great significance to design a reasonable parking lot dispatching strategy for the long-term operation benefit of the automatic parking lot.Aiming at the problem of automatic parking lot dispatching,the mathematical model of optimal dispatching and dispatching strategy are proposed from the following aspects:(1)The operation mechanism of the automatic parking lot is designed,and the automatic parking of the vehicle is transported by parking robots.The energy consumption optimization space of the automatic parking lot is analyzed.Based on the establishment of optimization model,a heuristic dynamic programming algorithm is proposed.The optimization strategy is established on the information of mass and time of parking.The mass and parking time of parked vehicles are normalized respectively.At the same time,the appropriate storage space for vehicles is arranged by combining the normalized information of both.Because the heuristic dynamic programming algorithm only considers the optimization strategy when the parking lot is empty,it does not deal with the change of parking lot status in real time.Using the idea of greedy algorithm for reference,the heuristic dynamic programming algorithm can revise the calculation results in real time by using the optimization results of greedy algorithm as the basis of dynamic adjustment.The scheduling strategy is further improved to reduce the operating energy consumption of the automated parking lot.(2)Because the dynamic adjustment algorithm only considers the number of parked vehicles in the parking lot,it does not consider the status of each parked vehicle.Based on reinforcement learning algorithm,the optimization problem of automatic parking lot is transformed into the optimal strategy solution problem under the framework of reinforcement learning algorithm.A scheduling optimization algorithm based on DQN is constructed by defining the state,value model and reward function of action in the operation of automatic parking lot.Using the structure of neural network to fit the value model,the real value of the value model is approached continuously through interaction with the environment and calculation of sample value.The parking lot scheduling strategy based on DQN can obtain better running results than the heuristic dynamic programming algorithm.(3)The traditional DQN algorithm is improved on the basis of DQN algorithm.In the aspect of sample data structure,the value of preserving state information and all actions in this state is different from the traditional form of transformation between different states.This kind of sample storage and training method can obtain more accurate value estimation in the early stage of algorithm training and avoid the value model falling into local optimal solution.The improved algorithm based on DQN has faster improvement ability in the early stage of training and obtains a better parking scheduling strategy.The experimental results show that the heuristic dynamic programming algorithm and the improved DQN algorithm can effectively reduce the running energy consumption of the automated parking lot,which is meaningful to the long-term operation of the automated parking lot.