Explainable Reinforcement Learning for Autonomous Multi-Agent Traffic Coordination

Abstract

Deep multi-agent reinforcement learning has become the dominant paradigm for coordinating networked traffic signal
controllers, with architectures ranging from centralized-critic actor-critic methods to attention-based network cooperation
reporting substantial delay and throughput gains over fixed-time and single-intersection baselines in simulation. Yet the same
neural-network opacity that gives these controllers their flexibility makes their decisions illegible to the traffic engineers,
regulators, and drivers who must ultimately trust them, motivating a growing explainable reinforcement learning (XRL)
literature aimed specifically at sequential decision-making systems. This article synthesizes three literatures usually read
separately: the multi-agent reinforcement learning architectures -- centralized critics, monotonic value decomposition,
counterfactual credit assignment -- that coordinate networked and vehicle-level traffic agents; the general explainable-AI and
explainable-RL techniques, from local surrogate models and saliency maps to policy distillation and counterfactual
explanation, developed to make learned policies legible; and the small but recent set of traffic-control systems, some published
as recently as 2025 and 2026, that build interpretability directly into the coordination architecture through pressure-grounded
rewards, attention weights, and explicit safety constraints rather than applying post-hoc explanation to an already-trained
black box. We trace a pattern in which learned attention mechanisms and constraint-satisfaction machinery, originally
introduced purely to improve coordination performance, now double as the field's most practically adopted explanation
mechanisms. We close by identifying open gaps -- the absence of a shared evaluation standard for RL explanation quality, the
unresolved trade-off between explanation fidelity and coordination performance, and the persistent gap between simulated
benchmarks and deployable, safety-certified infrastructure -- that the literature has not yet closed