Fetching the paper…

Reading the bibliography…

2025

LLM-Guided Probabilistic Program Induction for POMDP Model Estimation

Curtis, Aidan, Tang, Hao, Veloso, Thiago et al.

Original
PDF
Post on X

Understand

Partially Observable Markov Decision Processes (POMDPs) model decision making under uncertainty.

While there are many approaches to approximately solving POMDPs, we aim to address the problem of learning such models.
In particular, we are interested in a subclass of POMDPs wherein the components of the model, including the observation function, reward function, transition function, and initial state distribution function, can be modeled as low-complexity probabilistic graphical models in the form of a short probabilistic program.
Our strategy to learn these programs uses an LLM as a prior, generating candidate probabilistic programs that are then tested against the empirical distribution and adjusted through feedback.

LLM-Guided Probabilistic Program Induction for POMDP Model Estimation · Around

Built on

The witness algorithm: Solving partially observable markov decision processes
M. Littman · 1995
Earlier work this paper cites.
The complexity of policy evaluation for finite-horizon partially-observable markov decision processes
M. Mundhenk, J. Goldsmith, and E. Allender · 1997
Earlier work this paper cites.
Planning and acting in partially observable stochastic domains
L. P. Kaelbling, M. L. Littman, and A. R. Cassandra · 1998
Earlier work this paper cites.
Learning nonsingular phylogenies and hidden markov models
E. Mossel and S. Roch · 2005
Earlier work this paper cites.
Probabilistic Robotics (Intelligent Robotics and Autonomous Agents)
S. Thrun, W. Burgard, and D. Fox · 2005
Earlier work this paper cites.
Model-based reinforcement learning: A survey
Original
T. M. Moerland, J. Broekens, and C. M. Jonker · 2006
Earlier work this paper cites.
Sample-efficient reinforcement learning of undercomplete pomdps
Original
C. Jin, S. M. Kakade, A. Krishnamurthy, and Q. Liu · 2006
Earlier work this paper cites.
Dreamcoder: Growing generalizable, interpretable knowledge with wake-sleep bayesian program learning
Original
K. Ellis, C. Wong, M. I. Nye, M. Sablé-Meyer, L. Cary, L. Morales, L. B. Hewitt, A. Solar-Lezama, and J. B. Tenenbaum · 2006
Earlier work this paper cites.
Bayes-adaptive pomdps
S. Ross, B. Chaib-draa, and J. Pineau · 2007
Earlier work this paper cites.
Ff-replan: A baseline for probabilistic planning
S. W. Yoon, A. Fern, and R. Givan · 2007
Earlier work this paper cites.
Closing the learning-planning loop with predictive state representations, 2009
Original
B. Boots, S. M. Siddiqi, and G. J. Gordon · 2009
Earlier work this paper cites.
Parameter learning for pomdp spoken dialogue models
B. Thomson, F. Jurčíček, M. Gašić, S. Keizer, F. Mairesse, K. Yu, and S. Young · 2010
Earlier work this paper cites.
Monte-Carlo planning in large POMDPs
D. Silver and J. Veness · 2010
Earlier work this paper cites.
Apprenticeship learning for model parameters of partially observable environments, 2012
Original
T. Makino and J. Takeuchi · 2012
Earlier work this paper cites.
Church: a language for generative models
Original
N. D. Goodman, V. Mansinghka, D. M. Roy, K. A. Bonawitz, and J. B. Tenenbaum · 2012
Earlier work this paper cites.

Similar

Heuristic search value iteration for pomdps
Original
T. Smith and R. G. Simmons · 2012
Cited alongside, same era.
Heuristic search value iteration for pomdps
Original
T. Smith and R. G. Simmons · 2012
Cited alongside, same era.
Incremental pruning: A simple, fast, exact method for partially observable markov decision processes
Original
A. R. Cassandra, M. L. Littman, and N. L. Zhang · 2013
Cited alongside, same era.
Integrated task and motion planning in belief space
L. Kaelbling and T. Lozano-Perez · 2013
Cited alongside, same era.
Robotic manipulation of multiple objects as a pomdp
J. Pajarinen and V. Kyrki · 2015
Cited alongside, same era.

Then

From word models to world models: Translating from natural language to the probabilistic language of thought, 2023
Original
L. Wong, G. Grand, A. K. Lew, N. D. Goodman, V. K. Mansinghka, J. Andreas, and J. B. Tenenbaum · 2023
Later among the works it cites.
Minigrid & miniworld: Modular & customizable reinforcement learning environments for goal-oriented tasks
Original
M. Chevalier-Boisvert, B. Dai, M. Towers, R. de Lazcano, L. Willems, S. Lahlou, S. Pal, P. S. Castro, and J. Terry · 2023
Later among the works it cites.
Grounding dino: Marrying dino with grounded pre-training for open-set object detection
Original
S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, C. Li, J. Yang, H. Su, J. Zhu, et al · 2023
Later among the works it cites.
Automated statistical model discovery with language models, 2024
Original
M. Y. Li, E. B. Fox, and N. D. Goodman · 2024
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Human-level concept learning through probabilistic program induction

B. M. Lake, R. Salakhutdinov, and J. B. Tenenbaum · 2015

Cited alongside, same era.

Intention-aware autonomous driving decision-making in an uncontrolled intersection

W. Song, G. Xiong, and H. Chen · 2016

Cited alongside, same era.

POMCPOW: an online algorithm for pomdps with continuous state, action, and observation spaces

Z. Sunberg and M. J. Kochenderfer · 2017

Cited alongside, same era.

Asymmetric actor critic for image-based robot learning

Original

L. Pinto, M. Andrychowicz, P. Welinder, W. Zaremba, and P. Abbeel · 2017

Cited alongside, same era.

A decision-theoretic approach for the collaborative control of a smart wheelchair

M. Ghorbel, J. Pineau, R. Gourdeau, S. Javdani, and S. Srinivasa · 2018

Cited alongside, same era.

Pyro: Deep universal probabilistic programming

E. Bingham, J. P. Chen, M. Jankowiak, F. Obermeyer, N. Pradhan, T. Karaletsos, R. Singh, P. A. Szerlip, P. Horsfall, and N. D. Goodman · 2018

Cited alongside, same era.

Gen: a general-purpose probabilistic programming system with programmable inference

M. F. Cusumano-Towner, F. A. Saad, A. K. Lew, and V. K. Mansinghka · 2019

Cited alongside, same era.

Loose lips sink ships: Asking questions in battleship with language-informed program sampling, 2024

G. Grand, V. Pepe, J. Andreas, and J. B. Tenenbaum · 2024

Later among the works it cites.

Worldcoder, a model-based llm agent: Building world models by writing code and interacting with the environment, 2024

H. Tang, D. Key, and K. Ellis · 2024

Later among the works it cites.

Trust the proc3s: Solving long-horizon robotics problems with llms and constraint satisfaction, 2024

A. Curtis, N. Kumar, J. Cao, T. Lozano-Pérez, and L. P. Kaelbling · 2024

Later among the works it cites.

Large language models can plan your travels rigorously with formal verification tools

Y. Hao, Y. Chen, Y. Zhang, and C. Fan · 2024

Later among the works it cites.

Satlm: Satisfiability-aided language models using declarative prompting

Original

X. Ye, Q. Chen, I. Dillig, and G. Durrett · 2024

Later among the works it cites.

Partially observable task and motion planning with uncertainty and risk awareness, 2024

A. Curtis, G. Matheos, N. Gothoskar, V. Mansinghka, J. Tenenbaum, T. Lozano-Pérez, and L. P. Kaelbling · 2024

Later among the works it cites.

Code repair with llms gives an exploration-exploitation tradeoff

H. Tang, K. Hu, J. P. Zhou, S. Zhong, W.-L. Zheng, X. Si, and K. Ellis · 2024

Later among the works it cites.

Grounded sam: Assembling open-world models for diverse visual tasks, 2024

T. Ren, S. Liu, A. Zeng, J. Lin, K. Li, H. Cao, J. Chen, X. Huang, Y. Chen, F. Yan, Z. Zeng, H. Zhang, F. Li, J. Yang, H. Li, Q. Jiang, and L. Zhang · 2024

Later among the works it cites.

Visualpredicator: Learning abstract world models with neuro-symbolic predicates for robot planning, 2025

Y. Liang, N. Kumar, H. Tang, A. Weller, J. B. Tenenbaum, T. Silver, J. F. Henriques, and K. Ellis · 2025

Closest in time.

Polycam – capture and share 3d models, 2025

Polycam · 2025

Closest in time.