Publications

Bayesian Ensembles for Exploration in Deep Q-Learning

Pascal R. Van der Vaart, Neil Yorke-Smith, and Matthijs T. J. Spaan. Bayesian Ensembles for Exploration in Deep Q-Learning. In Proc. of the Adaptive and Learning Agents Workshop, 2024. Workshop at AAMAS

Download

pdf

Abstract

Exploration in reinforcement learning remains a difficult challenge. In order to drive exploration, ensembles with randomized prior functions have recently been popularized to quantify uncertainty in the value model. However these ensembles have no theoretical motivation why they should resemble the actual posterior. In this work, we view training ensembles from the perspective of Sequential Monte Carlo, a Monte Carlo method that approximates a sequence of distributions with a set of particles, and propose an algorithm that exploits both the practical flexibility of ensembles and theory of the Bayesian paradigm. We incorporate this method into a standard DQN agent and experimentally show qualitatively good uncertainty quantification and improved exploration capabilities over a regular ensemble.

BibTeX Entry

@InProceedings{VanDerVaart24ala,
  author =       {Van der Vaart, Pascal R. and Yorke-Smith, Neil and
                  Spaan, Matthijs T. J.},
  title =        {Bayesian Ensembles for Exploration in Deep
                  {Q}-Learning},
  year =         2024,
  booktitle =    {Proc. of the Adaptive and Learning Agents Workshop},
  note =         {Workshop at AAMAS}
}

Note: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

Generated by bib2html.pl (written by Patrick Riley) on Fri Aug 28, 2026 12:56:06 UTC