Online Planning in POMDPs with Self-Improving Simulators

Jinke He, Miguel Suau , Hendrik Baier, Michael Kaisers, Frans A. Oliehoek

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

1 Downloads (Pure)


How can we plan efficiently in a large and complex environment when the time budget is limited? Given the original simulator of the environment, which may be computationally very demanding, we propose to learn online an approximate but much faster simulator that improves over time. To plan reliably and efficiently while the approximate simulator is learning, we develop a method that adaptively decides which simulator to use for every simulation, based on a statistic that measures the accuracy of the approximate simulator. This allows us to use the approximate simulator to replace the original simulator for faster simulations when it is accurate enough under the current context, thus trading off simulation speed and accuracy. Experimental results in two large domains show that when integrated with POMCP, our approach allows to plan with improving efficiency over time.
Original languageEnglish
Title of host publicationProceedings of the Thirty-First International Joint Conference on Artificial Intelligence
EditorsLuc De Raedt
PublisherInternational Joint Conferences on Artificial Intelligence (IJCAI)
Number of pages7
ISBN (Electronic)978-1-956792-00-3
Publication statusPublished - 2022

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project

Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.


Dive into the research topics of 'Online Planning in POMDPs with Self-Improving Simulators'. Together they form a unique fingerprint.

Cite this