Research On Optimization Of Battery Replacement Decision For Shared E-bikes Based On Reinforcement Learning

In recent years,with the development of shared mobility,shared e-bikes have become one of the main means of transportation for users traveling medium or short distances.Considering the advanced technology of removable batteries and the flexibility and stability of the battery swapping mode,many shared mobility operators have launched removable battery swapping services.However,through literature review and field research,it was found that in actual operation,the existing battery swapping strategies are mainly divided into real-time triggering strategies and fixed-cycle battery swapping strategies.These strategies only make decisions based on the current status of the e-bikes and user needs,ignoring the future user needs that may arise.This may lead to low battery swapping efficiency,high swapping costs,and low battery utilization rates,which may affect the operating income of the enterprise and user satisfaction to some extent.Therefore,developing a scientifically reasonable battery swapping strategy based on the battery level of e-bikes in the region and the predicted demand in the near future is an urgent problem to be solved in the shared e-bikes industry.Based on literature review,on-site investigation and analysis of user cycling data,this thesis first extracted the problems in the decision-making process of battery replacement in the daily operation of e-bikes.On this basis,the thesis focuses on the research of the multi-period battery swapping decision optimization model considering stochastic demand,and the battery swapping optimization algorithm based on reinforcement learning.Firstly,this thesis uses real user cycling data of a shared e-bike operation company in Chengdu to analyze the spatiotemporal distribution characteristics of random demand and the dynamic changes of e-bike availability status.Based on this,a periodic centralized battery replacement strategy that can utilize future random demand information is proposed to reduce the operating cost of e-bike battery replacement.Secondly,a multi-period battery replacement decision-making optimization model based on Markov process is established based on the e-bike availability status and the spatiotemporal distribution characteristics of random demand,which details the state transition process of e-bike in the region.Then,in view of the complex structure of the optimization model,an intelligent dynamic battery replacement decision-making algorithm based on Q-learning is developed with the combination of reinforcement learning,and a dimension reduction method for state and action spaces is proposed to significantly improve the algorithm’s solving efficiency.Finally,the effectiveness and computational performance of the proposed model and algorithm are validated through numerical simulation experiments.The research of this thesis will provide reference and guidance for the daily battery exchange operation of shared e-bike in practice,helping enterprises to formulate more scientific and reasonable intelligent battery exchange decisions,which will contribute to improving the level of enterprise operation and management and achieving sustainable operation.