Newest papers

From Observation to Intervention: Mastering Bus Holding Control from Historical Uncontrolled Trajectories with Offline Reinforcement Learning

Minyu Shen, Xiaowen Bi, Weihua Gu, Zhen Xu, Yuezhong Wu, Feng Xiao, Lu Li · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.14632 · cited by 0 works (OpenAlex) · open-access PDF

German

Bus-Reisen sind sogar inhärent instabil---Busse, die ursprünglich auf gleichen Wegen entsandt werden, werden unweigerlich am Ende bündeln. Dieses Problem verschlimmert die Wartezeiten der Passagiere, schafft Lastunverhältnisse zwischen Bussen und untergräbt die Zuverlässigkeit des Dienstes. Reinforcement Learning (RL) hat sich als hochwirksam erwiesen, um Bus-Bündel zu mildern und traditionelle Modell-basierte Ansätze zu übertreffen. Jedoch erfordern bestehende RL-Ansätze Agenten, um kontinuierlich mit der Umwelt zu interagieren, was das Bus-System stören kann, bevor effektive Richtlinien gelernt werden. Dieses Papier schlägt eine Offline Graph-basierte Multi-Agent Actor-Critic (Off

Machine translation (Meta M2M100); the English original follows.

Abstract

Bus operations are inherently unstable---buses initially dispatched at equal headways will inevitably end up bunching up. This issue exacerbates passenger waiting times, creates load imbalances between buses, and undermines service reliability. Reinforcement learning (RL) has proven highly effective for mitigating bus bunching and outperforms traditional model-based approaches. However, existing RL approaches require agents to continuously interact with the environment, which potentially disrupts the bus system before effective policies are learned. This paper proposes an Offline Graph-based Multi-agent Actor-Critic (Offline-GMAC) method that trains bus holding policies using only readily available logged GPS data collected during normal uncontrolled operations, thereby eliminating the need for disruptive online interaction or exploration. The method employs general graph neural networks for multi-agent credit assignment. A key innovation of our method is a data augmentation scheme that synthesizes unperformed holding actions from historical uncontrolled bus trajectories. The scheme exploits the equivalence between stochastic link travel time delays and deliberate holding interventions. Under mild assumptions, we prove that the 1-Wasserstein distance between synthesized and target state transition distributions is bounded above by a term that scales linearly with the standard deviation of link travel times plus a tunable synthesis-dependent term. Evaluations using real-world bus data demonstrate that the proposed offline method achieves performance comparable to, and in some cases, even better than its online counterpart.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Reinforcement learning, Train, Exploit, Key (lock), Scheme (mathematics)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange