Publications

| Sorted by Date | Classified by Publication Type | Classified by Research Category |

Trust-Region Twisted Policy Improvement

Joery A. De Vries, Jinke He, Yaniv Oren, and Matthijs T. J. Spaan. Trust-Region Twisted Policy Improvement. In International Conference on Machine Learning, pp. 12901–12923, Proceedings of Machine Learning Research 267, 2025.

Download

pdf 

Abstract

Monte-Carlo tree search (MCTS) has driven many recent breakthroughs in deep reinforcement learning (RL). However, scaling MCTS to parallel compute has proven challenging in practice which has motivated alternative planners like sequential Monte-Carlo (SMC). Many of these SMC methods adopt particle filters for smoothing through a reformulation of RL as a policy inference problem. Yet, persisting design choices of these particle filters often conflict with the aim of online planning in RL, which is to obtain a policy improvement at the start of planning. Drawing inspiration from MCTS, we tailor SMC planners specifically to RL by improving data generation within the planner through constrained action sampling and explicit terminal state handling, as well as improving policy and value target estimation. This leads to our Trust-Region Twisted SMC (TRT-SMC), which shows improved runtime and sample-efficiency over baseline MCTS and SMC methods in both discrete and continuous domains.

BibTeX Entry

@InProceedings{DeVries25icml,
  author =       {De Vries, Joery A. and He, Jinke and Oren, Yaniv and
                  Spaan, Matthijs T. J.},
  title =        {Trust-Region Twisted Policy Improvement},
  booktitle =    {International Conference on Machine Learning},
  pages = 	 {12901--12923},
  volume = 	 {267},
  series = 	 {Proceedings of Machine Learning Research},
  year =         2025
}

Note: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

Generated by bib2html.pl (written by Patrick Riley) on Fri Aug 28, 2026 12:56:06 UTC