Fetching the paper…
Reading the bibliography…
Are world models a necessary ingredient for flexible, goal-directed behaviour, or is model-free learning sufficient? We provide a formal answer to this question, showing that any agent capable of generalizing to multi-step goal-directed tasks must have learned a predictive model of its environment.
Microcosmus: From anaximandros to paracelsus
Allers, R · 1944
Earlier work this paper cites.
Every good regulator of a system must be a model of that system
Conant, R. C. and Ross Ashby, W · 1970
Earlier work this paper cites.
The foundations of statistics
Savage, L. J · 1972
Earlier work this paper cites.
Judgment under uncertainty: Heuristics and biases: Biases in judgments reveal some heuristics of thinking under uncertainty
Tversky, A. and Kahneman, D · 1974
Earlier work this paper cites.
The temporal logic of programs
Pnueli, A · 1977
Earlier work this paper cites.
Perceptions as hypotheses
Gregory, R. L · 1980
Earlier work this paper cites.
Mental models: Towards a cognitive science of language, inference, and consciousness
Johnson-Laird, P. N · 1983
Earlier work this paper cites.
Empirical model-building and response surfaces
Box, G. E. and Draper, N. R · 1987
Earlier work this paper cites.
Intention, plans, and practical reason, 1987
Bratman, M · 1987
Earlier work this paper cites.
Toward a universal law of generalization for psychological science
Shepard, R. N · 1987
Earlier work this paper cites.
Intelligence without representation
Brooks, R. A · 1991
Earlier work this paper cites.
The social brain hypothesis
Dunbar, R. I · 1998
Earlier work this paper cites.
Algorithms for inverse reinforcement learning
Ng, A. Y., Russell, S., et al · 2000
Earlier work this paper cites.
The complexity of propositional linear temporal logics in simple cases
Demri, S. and Schnoebelen, P · 2002
Earlier work this paper cites.
Responsibility and blame: A structural-model approach
Chockler, H. and Halpern, J. Y · 2004
Earlier work this paper cites.
Reasoning about knowledge
Fagin, R., Halpern, J. Y., Moses, Y., and Vardi, M · 2004
Earlier work this paper cites.
Automated Planning: theory and practice
Ghallab, M., Nau, D., and Traverso, P · 2004
Earlier work this paper cites.
Goal inference as inverse planning
Baker, C. L., Tenenbaum, J. B., and Saxe, R. R · 2007
Earlier work this paper cites.
Theory of games and economic behavior: 60th anniversary commemorative edition
Von Neumann, J. and Morgenstern, O · 2007
Earlier work this paper cites.
Principles of model checking
Baier, C. and Katoen, J.-P · 2008
Earlier work this paper cites.
What to do and how to do it: Translating natural language directives into temporal and dynamic logic representation for goal management and action execution
Dzifcak, J., Scheutz, M., Baral, C., and Schermerhorn, P · 2009
Earlier work this paper cites.
The free-energy principle: a unified brain theory?
Friston, K · 2010
Earlier work this paper cites.
Thinking, fast and slow
Kahneman, D · 2011
Earlier work this paper cites.
Active inference and free energy
Friston, K · 2013
Earlier work this paper cites.
Goal setting theory, 1990
Locke, E. A. and Latham, G. P · 2013
Earlier work this paper cites.
Optimal control of markov decision processes with linear temporal logic constraints
Ding, X., Smith, S. L., Belta, C., and Rus, D · 2014
Earlier work this paper cites.
Markov decision processes: discrete stochastic dynamic programming
Puterman, M. L · 2014
Earlier work this paper cites.
Exploration versus exploitation in space, mind, and society
Hills, T. T., Todd, P. M., Lazer, D., Redish, A. D., and Couzin, I. D · 2015
Earlier work this paper cites.
Universal value function approximators
Schaul, T., Horgan, D., Gregor, K., and Silver, D · 2015
Earlier work this paper cites.
Understanding intermediate layers using linear classifier probes
Alain, G. and Bengio, Y · 2016
Earlier work this paper cites.
Towards resolving unidentifiability in inverse reinforcement learning
Amin, K. and Singh, S · 2016
Earlier work this paper cites.
Concrete problems in ai safety
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., and Mané, D · 2016
Earlier work this paper cites.
Good and safe uses of ai oracles
Armstrong, S. and O’Rorke, X · 2017
Earlier work this paper cites.
Building machines that learn and think like people
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., and Gershman, S. J · 2017
Cited alongside, same era.
Reinforcement learning with temporal logic rewards
Li, X., Vasile, C.-I., and Belta, C · 2017
Cited alongside, same era.
Environment-independent task specifications via gltl
Littman, M. L., Topcu, U., Fu, J., Isbell, C., Wen, M., and MacGlashan, J · 2017
Cited alongside, same era.
Imagination-augmented agents for deep reinforcement learning
Racanière, S., Weber, T., Reichert, D., Buesing, L., Guez, A., Jimenez Rezende, D., Puigdomènech Badia, A., Vinyals, O., Heess, N., Li, Y., et al · 2017
Cited alongside, same era.
Deep reinforcement learning in a handful of trials using probabilistic dynamics models
Chua, K., Calandra, R., McAllister, R., and Levine, S · 2018
Cited alongside, same era.
The evolution of agency: Behavioral organization from lizards to humans
Tomasello, M · 2022
Later among the works it cites.
React: Synergizing reasoning and acting in language models
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., and Cao, Y · 2022
Later among the works it cites.
The internal model principle
Baez, J · 2023
Later among the works it cites.
Towards bounding causal effects under markov equivalence
Bellot, A · 2023
Later among the works it cites.
Towards monosemanticity: Decomposing language models with dictionary learning
Bricken, T., Templeton, A., Batson, J., Chen, B., Jermyn, A., Conerly, T., Turner, N., Anil, C., Denison, C., Askell, A., et al · 2023
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Ha, D. and Schmidhuber, J · 2018
Cited alongside, same era.
Scalable agent alignment via reward modeling: a research direction
Leike, J., Krueger, D., Everitt, T., Martic, M., Maini, V., and Legg, S · 2018
Cited alongside, same era.
Theoretical impediments to machine learning with seven sparks from the causal revolution
Pearl, J · 2018
Cited alongside, same era.
Machine theory of mind
Rabinowitz, N., Perbet, F., Song, F., Zhang, C., Eslami, S. A., and Botvinick, M · 2018
Cited alongside, same era.
Reinforcement learning: an introduction
Sutton, R. S · 2018
Cited alongside, same era.
Ltl and beyond: Formal languages for reward function specification in reinforcement learning
Camacho, A., Icarte, R. T., Klassen, T. Q., Valenzano, R. A., and McIlraith, S. A · 2019
Cited alongside, same era.
Challenges of real-world reinforcement learning
Dulac-Arnold, G., Mankowitz, D., and Hester, T · 2019
Cited alongside, same era.
Brohan, A., Brown, N., Carbajal, J., Chebotar, Y., Chen, X., Choromanski, K., Ding, T., Driess, D., Dubey, A., Finn, C., et al · 2023
Later among the works it cites.
Palm-e: An embodied multimodal language model
Driess, D., Xia, F., Sajjadi, M. S., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., Yu, T., et al · 2023
Later among the works it cites.
Mastering diverse domains through world models
Hafner, D., Pasukonis, J., Ba, J., and Lillicrap, T · 2023
Later among the works it cites.
Reasoning with language model is planning with world model
Hao, S., Gu, Y., Ma, H., Hong, J. J., Wang, Z., Wang, D. Z., and Hu, Z · 2023
Later among the works it cites.
Towards a mechanistic interpretation of multi-step reasoning capabilities of language models
Hou, Y., Li, J., Fei, Y., Stolfo, A., Zhou, W., Zeng, G., Bosselut, A., and Sachan, M · 2023
Later among the works it cites.
Scaling deep learning for materials discovery
Merchant, A., Batzner, S., Schoenholz, S. S., Aykol, M., Cheon, G., and Cubuk, E. D · 2023
Later among the works it cites.
Instructing goal-conditioned reinforcement learning agents with temporal logic objectives
Qiu, W., Mao, W., and Zhu, H · 2023
Later among the works it cites.
Voyager: An open-ended embodied agent with large language models
Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., and Anandkumar, A · 2023
Later among the works it cites.
Honesty is the best policy: defining and mitigating ai deception
Ward, F., Toni, F., Belardinelli, F., and Everitt, T · 2023
Later among the works it cites.
Fixing the good regulator theorem
Wentworth, J · 2023
Later among the works it cites.
Transfer learning in deep reinforcement learning: A survey
Zhu, Z., Lin, K., Jain, A. K., and Zhou, J · 2023
Later among the works it cites.
Accurate structure prediction of biomolecular interactions with alphafold 3
Abramson, J., Adler, J., Dunger, J., Evans, R., Green, T., Pritzel, A., Ronneberger, O., Willmore, L., Ballard, A. J., Bambrick, J., et al · 2024
Later among the works it cites.
Can a bayesian oracle prevent harm from an agent?
Bengio, Y., Cohen, M. K., Malkin, N., MacDermott, M., Fornasiere, D., Greiner, P., and Kaddar, Y · 2024
Later among the works it cites.
π 0 \pi_{0} : A vision-language-action flow model for general robot control
Black, K., Brown, N., Driess, D., Esmail, A., Equi, M., Finn, C., Fusai, N., Groom, L., Hausman, K., Ichter, B., et al · 2024
Later among the works it cites.
Towards guaranteed safe ai: A framework for ensuring robust and reliable ai systems
Dalrymple, D., Skalse, J., Bengio, Y., Russell, S., Tegmark, M., Seshia, S., Omohundro, S., Szegedy, C., Goldhaber, B., Ammann, N., et al · 2024
Later among the works it cites.
A survey on interpretable reinforcement learning
Glanois, C., Weng, P., Zimmer, M., Li, D., Yang, T., Hao, J., and Liu, W · 2024
Later among the works it cites.
Halpern, J. Y. and Piermont, E · 2024
Later among the works it cites.
Deepltl: Learning to efficiently satisfy complex ltl specifications
Jackermeier, M. and Abate, A · 2024
Later among the works it cites.
Emergent world models and latent variable estimation in chess-playing language models
Karvonen, A · 2024
Later among the works it cites.
Instructing goal-conditioned reinforcement learning agents with temporal logic objectives
Qiu, W., Mao, W., and Zhu, H · 2024
Later among the works it cites.
Scaling instructable agents across many simulated worlds
Raad, M. A., Ahuja, A., Barros, C., Besse, F., Bolt, A., Bolton, A., Brownfield, B., Buttimore, G., Cant, M., Chakera, S., et al · 2024
Later among the works it cites.
Robust agents learn causal world models
Richens, J. and Everitt, T · 2024
Later among the works it cites.
The reasons that agents act: Intention and instrumental goals
Ward, F. R., MacDermott, M., Belardinelli, F., Toni, F., and Everitt, T · 2024
Later among the works it cites.
Superintelligent agents pose catastrophic risks: Can scientist ai offer a safer path?
Bengio, Y., Cohen, M., Fornasiere, D., Ghosn, J., Greiner, P., MacDermott, M., Mindermann, S., Oberman, A., Richardson, J., Richardson, O., et al · 2025
Closest in time.
Interpreting emergent planning in model-free reinforcement learning
Bush, T., Chung, S., Anwar, U., Garriga-Alonso, A., and Krueger, D · 2025
Closest in time.
Mona: Myopic optimization with non-myopic approval can mitigate multi-step reward hacking
Farquhar, S., Varma, V., Lindner, D., Elson, D., Biddulph, C., Goodfellow, I., and Shah, R · 2025
Closest in time.
Deepltl: Learning to efficiently satisfy complex ltl specifications for multi-task rl
Jackermeier, M. and Abate, A · 2025
Closest in time.