2020

Learning to Generate Levels From Nothing

Bontrager, Philip, Togelius, Julian

Understand

Machine learning for procedural content generation has recently become an active area of research.

Levels vary in both form and function and are mostly unrelated to each other across games.
This has made it difficult to assemble suitably large datasets to bring machine learning to level design in the same way as it's been used for image generation.
Here we propose Generative Playing Networks which design levels for itself to play.

Built on

V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. Lillicrap, T. Harley, D. Silver, and K. Kavukcuoglu, “Asynchronous methods for deep reinforcement learning,” in International conference on machine learning , 2016, pp. 1928–1937
1937
Earlier work this paper cites.
J. Togelius and J. Schmidhuber, “An experiment in automatic game design,” in 2008 IEEE Symposium On Computational Intelligence and Games . IEEE, 2008, pp. 111–118
2008
Earlier work this paper cites.
J. Togelius, G. N. Yannakakis, K. O. Stanley, and C. Browne, “Search-based procedural content generation: A taxonomy and survey,” IEEE Transactions on Computational Intelligence and AI in Games , vol. 3, no. 3, pp. 172–186, 2011
2011
Earlier work this paper cites.
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in neural information processing systems , 2014, pp. 2672–2680
2014
Earlier work this paper cites.
D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” stat , vol. 1050, p. 1, 2014
2014
Earlier work this paper cites.
A. Nguyen, J. Yosinski, and J. Clune, “Deep neural networks are easily fooled: High confidence predictions for unrecognizable images,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2015, pp. 427–436
2015
Earlier work this paper cites.
A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” arXiv preprint arXiv:1511.06434 , 2015
Original
2015
Earlier work this paper cites.
N. Shaker, J. Togelius, and M. J. Nelson, Procedural content generation in games . Springer, 2016
2016
Earlier work this paper cites.
A. Nguyen, A. Dosovitskiy, J. Yosinski, T. Brox, and J. Clune, “Synthesizing the preferred inputs for neurons in neural networks via deep generator networks,” in Advances in neural information processing systems , 2016, pp. 3387–3395
2016
Earlier work this paper cites.

Similar

G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, “Openai gym,” arXiv preprint arXiv:1606.01540 , 2016
Original
2016
Cited alongside, same era.
W. Shi, J. Caballero, F. Huszár, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang, “Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 1874–1883
2016
Cited alongside, same era.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778
2016
Cited alongside, same era.
K. Cobbe, O. Klimov, C. Hesse, T. Kim, and J. Schulman, “Quantifying generalization in reinforcement learning,” arXiv preprint arXiv:1812.02341 , 2018
Original
2018
Cited alongside, same era.
C. Zhang, O. Vinyals, R. Munos, and S. Bengio, “A study on overfitting in deep reinforcement learning,” arXiv preprint arXiv:1804.06893 , 2018
Original
2018
Cited alongside, same era.
A. Summerville, S. Snodgrass, M. Guzdial, C. Holmgård, A. K. Hoover, A. Isaksen, A. Nealen, and J. Togelius, “Procedural content generation via machine learning (pcgml),” IEEE Transactions on Games , vol. 10, no. 3, pp. 257–270, 2018
2018
Cited alongside, same era.
P. Bontrager, A. Roy, J. Togelius, N. Memon, and A. Ross, “Deepmasterprints: Generating masterprints for dictionary attacks via latent variable evolution,” in 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS) . IEEE, 2018, pp. 1–9
2018
Cited alongside, same era.
V. Volz, J. Schrum, J. Liu, S. M. Lucas, A. Smith, and S. Risi, “Evolving mario levels in the latent space of a deep convolutional generative adversarial network,” in Proceedings of the Genetic and Evolutionary Computation Conference , 2018, pp. 221–228
2018
Cited alongside, same era.
R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction . MIT press, 2018
2018
Cited alongside, same era.

Then

R. R. Torrado, P. Bontrager, J. Togelius, J. Liu, and D. Perez-Liebana, “Deep reinforcement learning for general video game ai,” in 2018 IEEE Conference on Computational Intelligence and Games (CIG) . IEEE, 2018, pp. 1–8
2018
Later among the works it cites.
S. Risi and J. Togelius, “Procedural content generation: From automatically generating game levels to increasing generality in machine learning,” arXiv preprint arXiv:1911.13071 , 2019
Original
2019
Later among the works it cites.
N. Justesen, R. R. Torrado, P. Bontrager, A. Khalifa, J. Togelius, and S. Risi, “Illuminating generalization in deep reinforcement learning through procedural level generation,” in AAAI Workshop on Reinforcement Learning in Games , 2019
2019
Later among the works it cites.
R. Wang, J. Lehman, J. Clune, and K. O. Stanley, “Poet: open-ended coevolution of environments and their optimized solutions,” in Proceedings of the Genetic and Evolutionary Computation Conference , 2019, pp. 142–151
2019
Later among the works it cites.
N. Justesen, P. Bontrager, J. Togelius, and S. Risi, “Deep learning for video game playing,” IEEE Transactions on Games , 2019
2019
Later among the works it cites.
D. Perez-Liebana, J. Liu, A. Khalifa, R. D. Gaina, J. Togelius, and S. M. Lucas, “General video game ai: A multitrack framework for evaluating agents, games, and content generation algorithms,” IEEE Transactions on Games , vol. 11, no. 3, pp. 195–214, 2019
2019
Later among the works it cites.
A. Khalifa, P. Bontrager, S. Earle, and J. Togelius, “Pcgrl: Procedural content generation via reinforcement learning,” arXiv preprint arXiv:2001.09212 , 2020
Original
2020
Closest in time.
M. Dennis, N. Jaques, E. Vinitsky, A. Bayen, S. Russell, A. Critch, and S. Levine, “Emergent complexity and zero-shot transfer via unsupervised environment design,” arXiv preprint arXiv:2012.02096 , 2020
Original
2020
Closest in time.
L. Gisslén, A. Eakins, C. Gordillo, J. Bergdahl, and K. Tollmar, “Adversarial reinforcement learning for procedural content generation,” arXiv preprint arXiv:2103.04847 , 2021
Original
2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…