2018

Investigating Human Priors for Playing Video Games

Dubey, Rachit, Agrawal, Pulkit, Pathak, Deepak et al.

Understand

What makes humans so good at solving seemingly complex video games? Unlike computers, humans bring in a great deal of prior knowledge about the world, enabling efficient decision making.

This paper investigates the role of human priors for solving video games.
Given a sample game, we conduct a series of ablation studies to quantify the importance of various priors on human performance.
We do this by modifying the video game environment to systematically mask different types of visual information that could be used by humans as priors.

Built on

Principles of object perception
Spelke, E. S · 1990
Earlier work this paper cites.
How do infants learn about the physical world?
Baillargeon, R · 1994
Earlier work this paper cites.
Representation
Mandler, J. M · 1998
Earlier work this paper cites.
Computational principles of movement neuroscience
Wolpert, D. M. and Ghahramani, Z · 2000
Earlier work this paper cites.
Categorization in infancy
Mareschal, D. and Quinn, P. C · 2001
Earlier work this paper cites.
Infants’ physical world
Baillargeon, R · 2004
Earlier work this paper cites.
Cortical substrates for exploratory decisions in humans
Daw, N. D., O’Doherty, J. P., Dayan, P., Seymour, B., and Dolan, R. J · 2006
Earlier work this paper cites.
Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration
Cohen, J. D., McClure, S. M., and Angela, J. Y · 2007
Earlier work this paper cites.
Core knowledge
Spelke, E. S. and Kinzler, K. D · 2007
Earlier work this paper cites.
An object-oriented representation for efficient reinforcement learning
Diuk, C., Cohen, A., and Littman, M. L · 2008
Earlier work this paper cites.
The origin of concepts
Carey, S · 2009
Earlier work this paper cites.

Similar

Five-month-old infants have different expectations for solids and liquids
Hespos, S. J., Ferry, A. L., and Rips, L. J · 2009
Cited alongside, same era.
Developmental changes in visual object recognition between 18 and 24 months of age
Pereira, A. F. and Smith, L. B · 2009
Cited alongside, same era.
An object-oriented representation for efficient reinforcement learning
Wasser, C. G. D · 2010
Cited alongside, same era.
The double-edged sword of pedagogy: Instruction limits spontaneous exploration and discovery
Bonawitz, E., Shafto, P., Gweon, H., Goodman, N. D., Spelke, E., and Schulz, L · 2011
Cited alongside, same era.
A comparison of human and agent reinforcement learning in partially observable domains
Doshi-Velez, F. and Ghahramani, Z · 2011
Cited alongside, same era.
The nature of belief-directed exploratory choice in human decision-making
Knox, W. B., Otto, A. R., Stone, P., and Love, B · 2012
Cited alongside, same era.
Cliff or step? Posture-specific learning at the edge of a drop-off
Kretch, K. S. and Adolph, K. E · 2013
Cited alongside, same era.
The ecological approach to visual perception: classic edition
Gibson, J. J · 2014
Cited alongside, same era.
When children are better (or at least more open-minded) learners than adults: Developmental differences in learning the forms of causal relationships
Lucas, C. G., Bridgers, S., Griffiths, T. L., and Gopnik, A · 2014
Cited alongside, same era.
Novelty and inductive generalization in human reinforcement learning
Gershman, S. J. and Niv, Y · 2015
Cited alongside, same era.

Then

Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al · 2015
Later among the works it cites.
Unifying count-based exploration and intrinsic motivation
Bellemare, M., Srinivasan, S., Ostrovski, G., Schaul, T., Saxton, D., and Munos, R · 2016
Later among the works it cites.
Continuous deep q-learning with model-based acceleration
Gu, S., Lillicrap, T., Sutskever, I., and Levine, S · 2016
Later among the works it cites.
Building machines that learn and think like people
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., and Gershman, S. J · 2016
Later among the works it cites.
Asynchronous methods for deep reinforcement learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., and Kavukcuoglu, K · 2016
Later among the works it cites.
Schema networks: Zero-shot transfer with a generative causal model of intuitive physics
Kansky, K., Silver, T., Mély, D. A., Eldawy, M., Lázaro-Gredilla, M., Lou, X., Dorfman, N., Sidor, S., Phoenix, S., and George, D · 2017
Later among the works it cites.
Deep transfer in reinforcement learning by language grounding
Original
Narasimhan, K., Barzilay, R., and Jaakkola, T · 2017
Later among the works it cites.
Value prediction network
Oh, J., Singh, S., and Lee, H · 2017
Later among the works it cites.
Curiosity-driven exploration by self-supervised prediction
Pathak, D., Agrawal, P., Efros, A. A., and Darrell, T · 2017
Later among the works it cites.
Human learning in atari
Tsividis, P. A., Pouncy, T., Xu, J. L., Tenenbaum, J. B., and Gershman, S. J · 2017
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…