Reinforcement Learning-Based Bus Holding for High-Frequency Services

Since the bus holding problem is an operational control problem, bus holding decisions should be made in realtime. For this reason, common bus holding approaches, such as the one-headway-based holding, focus on computationally inexpensive, rule-based techniques that try to minimize the deviation of the actual headways from the planned ones. Nevertheless, rule-based methods optimize the system locally without considering the full effect of the bus holding decisions to future trips or other performance indicators. For this reason, this work introduces a Reinforcement Learning approach which is capable of making holistic bus holding decisions in realtime after the completion of a training period. The proposed approach is trained in a circular bus line in Singapore using 400 episodes (where an episode is one day of operations) and evaluated using 200 episodes demonstrating a significant improvement in scenarios with strong travel time disturbances and a slight improvement in scenarios with low travel time variations.