Publications

| Sorted by Date | Classified by Publication Type | Classified by Research Category |

Twice-Sequential Monte Carlo for Tree Search

Yaniv Oren, Joery A. De Vries, Pascal R. Van der Vaart, Matthijs T. J. Spaan, and Wendelin Böhmer. Twice-Sequential Monte Carlo for Tree Search. In International Conference on Machine Learning, 2026.

Download

pdf 

Abstract

Model-based reinforcement learning (RL) methods that leverage search are responsible for many milestone breakthroughs in RL. Sequential Monte Carlo (SMC) recently emerged as an alternative to the Monte Carlo Tree Search (MCTS) algorithm which drove these breakthroughs. SMC is easier to parallelize and more suitable to GPU acceleration. However, it also suffers from large variance and path degeneracy which prevent it from scaling well with increased search depth, i.e., increased sequential compute. To address these problems, we introduce Twice Sequential Monte Carlo Tree Search (TSMCTS). Across discrete and continuous environments TSMCTS outperforms the SMC baseline as well as a popular modern version of MCTS as a policy improvement operator, scales favorably with sequential compute, reduces estimator variance and mitigates the effects of path degeneracy while retaining the properties that make SMC natural to parallelize.

BibTeX Entry

@InProceedings{Oren26tsmcts,
  author =       {Oren, Yaniv and De Vries, Joery A. and Van der
                  Vaart, Pascal R. and Spaan, Matthijs T. J. and
                  B{\"o}hmer, Wendelin},
  title =        {Twice-Sequential {M}onte {C}arlo for Tree Search},
  booktitle =    {International Conference on Machine Learning},
  year =         2026
}

Note: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

Generated by bib2html.pl (written by Patrick Riley) on Fri Aug 28, 2026 12:56:06 UTC