Publications

| Sorted by Date | Classified by Publication Type | Classified by Research Category |

How Ensembles of Distilled Policies Improve Generalisation in Reinforcement Learning

Max Weltevrede, Moritz A. Zanger, Matthijs T. J. Spaan, and Wendelin Böhmer. How Ensembles of Distilled Policies Improve Generalisation in Reinforcement Learning. In Advances in Neural Information Processing Systems, pp. 135932–135972, 38, 2025.

Download

pdf 

Abstract

In the zero-shot policy transfer setting in reinforcement learning, the goal is to train an agent on a fixed set of training environments so that it can generalise to similar, but unseen, testing environments. Previous work has shown that policy distillation after training can sometimes produce a policy that outperforms the original in the testing environments. However, it is not yet entirely clear why that is, or what data should be used to distil the policy. In this paper, we prove, under certain assumptions, a generalisation bound for policy distillation after training. The theory provides two practical insights: for improved generalisation, you should 1) train an ensemble of distilled policies, and 2) distil it on as much data from the training environments as possible. We empirically verify that these insights hold in more general settings, when the assumptions required for the theory no longer hold. Finally, we demonstrate that an ensemble of policies distilled on a diverse dataset can generalise significantly better than the original agent.

BibTeX Entry

@InProceedings{Weltevrede25ensembles,
  author =       {Max Weltevrede and Moritz A. Zanger and Matthijs
                  T. J. Spaan and Wendelin B{\"o}hmer},
  title =        {How Ensembles of Distilled Policies Improve
                  Generalisation in Reinforcement Learning},
  booktitle =    {Advances in Neural Information Processing Systems},
  pages =        {135932--135972},
  volume =       38,
  year =         2025
}

Note: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

Generated by bib2html.pl (written by Patrick Riley) on Fri Aug 28, 2026 12:56:06 UTC