A stochastic scheduling, allocation, and inventory replenishment problem for battery swap stations

Electric vehicles and drones promise to transform transportation systems and supply chains. However, long recharge times and battery degradation inhibit adoption. To overcome these barriers, swap stations enable quick battery exchange. We introduce a stochastic scheduling, allocation, and inventory replenishment problem which determines the charging, discharging, and replacement decisions at a swap station over time. The decisions are complex because recharging is necessary for short-term operation but causes degradation and the need for future replacement. We model the problem as a Markov Decision Process, solve it using backward induction, and show that the problem suffers from the curses of dimensionality. Hence, we propose two approximate methods, a heuristic benchmark policy and a reinforcement learning method, which provide high-quality solutions. Using a designed experiment, we deduce effective operational insights.