2018

BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning

Chevalier-Boisvert, Maxime, Bahdanau, Dzmitry, Lahlou, Salem et al.

Understand

Allowing humans to interactively train artificial agents to understand language instructions is desirable for both practical and scientific reasons, but given the poor data efficiency of the current learning methods, this goal may require substantial research efforts.

Here, we introduce the BabyAI research platform to support investigations towards including humans in the loop for grounded language learning.
The BabyAI platform comprises an extensible suite of 19 levels of increasing difficulty.
The levels gradually lead the agent towards acquiring a combinatorially rich synthetic language which is a proper subset of English.

Built on

Long Short-Term Memory
Hochreiter, S. and Schmidhuber, J. (1997) · 1997
Earlier work this paper cites.
Design and Use Paradigms for Gazebo, An Open-Source Multi-Robot Simulator
Koenig, N. and Howard, A. (2004) · 2004
Earlier work this paper cites.
Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning)
Rasmussen, C. E. and Williams, C. K. I. (2005) · 2005
Earlier work this paper cites.
Walk the Talk: Connecting Language, Knowledge, Action in Route Instructions
Macmahon, M., Stankiewicz, B., and Kuipers, B. (2006) · 2006
Earlier work this paper cites.
Maximum Entropy Inverse Reinforcement Learning
Ziebart, B. D., Maas, A., Bagnell, J. A., and Dey, A. K. (2008) · 2008
Earlier work this paper cites.
Curriculum learning
Bengio, Y., Louradour, J., Collobert, R., and Weston, J. (2009) · 2009
Earlier work this paper cites.
Search-based structured prediction
Daumé Iii, H., Langford, J., and Marcu, D. (2009) · 2009
Earlier work this paper cites.
Self-Paced Learning for Latent Variable Models
Kumar, M. P., Packer, B., and Koller, D. (2010) · 2010
Earlier work this paper cites.
Learning to Interpret Natural Language Navigation Instructions from Observations
Chen, D. L. and Mooney, R. J. (2011) · 2011
Earlier work this paper cites.
Scikit-learn: machine learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011) · 2011
Earlier work this paper cites.
A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Ross, S., Gordon, G., and Bagnell, D. (2011) · 2011
Earlier work this paper cites.
Understanding Natural Language Commands for Robotic Navigation and Mobile Manipulation
Tellex, S., Kollar, T., Dickerson, S., Walter, M. R., Banerjee, A. G., Teller, S., and Roy, N. (2011) · 2011
Earlier work this paper cites.
A Bayesian Approach for Policy Learning from Trajectory Preference Queries
Wilson, A., Fern, A., and Tadepalli, P. (2012) · 2012
Earlier work this paper cites.
Weakly supervised learning of semantic parsers for mapping instructions to actions
Artzi, Y. and Zettlemoyer, L. (2013) · 2013
Earlier work this paper cites.
The Arcade Learning Environment: An Evaluation Platform for General Agents
Bellemare, M. G., Naddaf, Y., Veness, J., and Bowling, M. (2013) · 2013
Earlier work this paper cites.

Similar

Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bougares, F., Schwenk, H., and Bengio, Y. (2014) · 2014
Cited alongside, same era.
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., and Le, Q. V. (2014) · 2014
Cited alongside, same era.
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., and Bengio, Y. (2015) · 2015
Cited alongside, same era.
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. and Szegedy, C. (2015) · 2015
Cited alongside, same era.
Adam: A Method for Stochastic Optimization
Kingma, D. P. and Ba, J. (2015) · 2015
Cited alongside, same era.
High-Dimensional Continuous Control Using Generalized Advantage Estimation
Schulman, J., Moritz, P., Levine, S., Jordan, M., and Abbeel, P. (2015) · 2015
Cited alongside, same era.
MazeBase: A Sandbox for Learning from Games
Original
Sukhbaatar, S., Szlam, A., Synnaeve, G., Chintala, S., and Fergus, R. (2015) · 2015
Cited alongside, same era.
Learning to Execute
Zaremba, W. and Sutskever, I. (2015) · 2015
Cited alongside, same era.
Neural Module Networks
Andreas, J., Rohrbach, M., Darrell, T., and Klein, D. (2016) · 2016
Cited alongside, same era.
Hybrid computing using a neural network with dynamic external memory
Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I., Grabska-Barwińska, A., Colmenarejo, S. G., Grefenstette, E., Ramalho, T., Agapiou, J., Badia, A. P., Hermann, K. M., Zwols, Y., Ostrovski, G., Cain, A., King, H., Summerfield, C., Blunsom, P., Kavukcuoglu, K., and Hassabis, D. (2016) · 2016
Cited alongside, same era.
ViZDoom: A Doom-based AI research platform for visual reinforcement learning
Kempka, M., Wydmuch, M., Runc, G., Toczek, J., and Jaskowski, W. (2016) · 2016
Cited alongside, same era.
Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequences
Mei, H., Bansal, M., and Walter, M. R. (2016) · 2016
Cited alongside, same era.

Then

Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Original
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., and others (2016) · 2016
Later among the works it cites.
Deep reinforcement learning from human preferences
Christiano, P., Leike, J., Brown, T. B., Martic, M., Legg, S., and Amodei, D. (2017) · 2017
Later among the works it cites.
Grounded Language Learning in a Simulated 3d World
Original
Hermann, K. M., Hill, F., Green, S., Wang, F., Faulkner, R., Soyer, H., Szepesvari, D., Czarnecki, W. M., Jaderberg, M., Teplyashin, D., Wainwright, M., Apps, C., Hassabis, D., and Blunsom, P. (2017) · 2017
Later among the works it cites.
AI2-THOR: An Interactive 3d Environment for Visual AI
Original
Kolve, E., Mottaghi, R., Gordon, D., Zhu, Y., Gupta, A., and Farhadi, A. (2017) · 2017
Later among the works it cites.
FiLM: Visual Reasoning with a General Conditioning Layer
Perez, E., Strub, F., de Vries, H., Dumoulin, V., and Courville, A. (2017) · 2017
Later among the works it cites.
Proximal Policy Optimization Algorithms
Original
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017) · 2017
Later among the works it cites.
Deep TAMER: Interactive Agent Shaping in High-Dimensional State Spaces
Warnell, G., Waytowich, N., Lawhern, V., and Stone, P. (2017) · 2017
Later among the works it cites.
Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments
Anderson, P., Wu, Q., Teney, D., Bruce, J., Johnson, M., Sünderhauf, N., Reid, I., Gould, S., and Hengel, A. v. d. (2018) · 2018
Closest in time.
Learning to Understand Goal Specifications by Modelling Reward
Bahdanau, D., Hill, F., Leike, J., Hughes, E., Hosseini, A., Kohli, P., and Grefenstette, E. (2018) · 2018
Closest in time.
Gated-Attention Architectures for Task-Oriented Language Grounding
Chaplot, D. S., Sathyendra, K. M., Pasumarthi, R. K., Rajagopal, D., and Salakhutdinov, R. (2018) · 2018
Closest in time.
IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
Espeholt, L., Soyer, H., Munos, R., Simonyan, K., Mnih, V., Ward, T., Doron, Y., Firoiu, V., Harley, T., Dunning, I., Legg, S., and Kavukcuoglu, K. (2018) · 2018
Closest in time.
Learning to Parse Natural Language to Grounded Reward Functions with Weak Supervision
Williams, E. C., Gopalan, N., Rhee, M., and Tellex, S. (2018) · 2018
Closest in time.
Building Generalizable Agents with a Realistic and Rich 3d Environment
Original
Wu, Y., Wu, Y., Gkioxari, G., and Tian, Y. (2018) · 2018
Closest in time.
Interactive Grounded Language Acquisition and Generalization in 2d Environment
Yu, H., Zhang, H., and Xu, W. (2018) · 2018
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…