On Generating Explanations for Reinforcement Learning Policies: An Empirical Study

IEEE Control Systems Letters, The 2025 American Control Conference (ACC)

2024

Abstract

Explaining reinforcement learning policies is important for deploying them in real-world scenarios. We introduce a set of linear temporal logic formulae designed to provide such explanations, and an algorithm for searching through those formulae for the one that best explains a given policy. Our key idea is to compare action distributions from the target policy with those from policies optimized for candidate explanations. This comparison provides more insight into the target policy than existing methods and avoids inference of “catch-all” explanations. We demonstrate our method in a simulated game of capture-the-flag, a car-parking environment, and a robot navigation task.

Poster

Citation

@article{yuasa_generating_2024,
  title = {On {{Generating Explanations}} for {{Reinforcement Learning Policies}}: {{An Empirical Study}}},
  shorttitle = {On {{Generating Explanations}} for {{Reinforcement Learning Policies}}},
  author = {Yuasa, Mikihisa and Tran, Huy T. and Sreenivas, Ramavarapu S.},
  year = {2024},
  journal = {IEEE Control Systems Letters},
  volume = {8},
  pages = {3027--3032}
}