2019

Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learning

Nguyen, Khanh, Daumé III, Hal

Understand

Mobile agents that can leverage help from humans can potentially accomplish more complex tasks than they could entirely on their own.

We develop "Help, Anna!" (HANNA), an interactive photo-realistic simulator in which an agent fulfills object-finding tasks by requesting and interpreting natural language-and-vision assistance.
An agent solving tasks in a HANNA environment can leverage simulated human assistants, called ANNA (Automatic Natural Navigation Assistants), which, upon request, provide natural language and visual instructions to direct the agent towards the goals.
To address the HANNA problem, we develop a memory-augmented neural agent that hierarchically models multiple levels of decision-making, and an imitation learning algorithm that teaches the agent to avoid repeating past mistakes while simultaneously predicting its own chances of making future progress.

Built on

Learning to speak and act in a fantasy text adventure game
Original
Jack Urbanek, Angela Fan, Siddharth Karamcheti, Saachi Jain, Samuel Humeau, Emily Dinan, Tim Rocktäschel, Douwe Kiela, Arthur Szlam, and Jason Weston. 2019 · 1903
Earlier work this paper cites.
Rerere: Remote embodied referring expressions in real indoor environments
Original
Yuankai Qi, Qi Wu, Peter Anderson, Marco Liu, Chunhua Shen, and Anton van den Hengel. 2019 · 1904
Earlier work this paper cites.
From video game to real robot: The transfer between action spaces
Original
Janne Karttunen, Anssi Kanervisto, Ville Hautamäki, and Ville Kyrki. 2019 · 1905
Earlier work this paper cites.
Neural text generation with unlikelihood training
Original
Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, and Jason Weston. 2019 · 1908
Earlier work this paper cites.
Interactive robot task training through dialog and demonstration
Paul E Rybski, Kevin Yoon, Jeremy Stolarz, and Manuela M Veloso. 2007 · 2007
Earlier work this paper cites.
Search-based structured prediction
Hal Daumé III, John Langford, and Daniel Marcu. 2009 · 2009
Earlier work this paper cites.
A reduction of imitation learning and structured prediction to no-regret online learning
Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. 2011 · 2011
Earlier work this paper cites.
Imitation learning by coaching
He He, Jason Eisner, and Hal Daumé III. 2012 · 2012
Earlier work this paper cites.
Mujoco: A physics engine for model-based control
Emanuel Todorov, Tom Erez, and Yuval Tassa. 2012 · 2012
Earlier work this paper cites.
Playing atari with deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013 · 2013
Earlier work this paper cites.
Tell me dave: Contextsensitive grounding of natural language to mobile manipulation instructions
Dipendra K Misra, Jaeyong Sung, Kevin Lee, and Ashutosh Saxena. 2014 · 2014
Earlier work this paper cites.
Learning goal-oriented hierarchical tasks from situated interactive instruction
Shiwali Mohan and John Laird. 2014 · 2014
Earlier work this paper cites.
Reinforcement and imitation learning via interactive no-regret learning
Original
Stephane Ross and J Andrew Bagnell. 2014 · 2014
Earlier work this paper cites.
Back to the blocks world: Learning new actions through situated human-robot dialogue
Lanbo She, Shaohua Yang, Yu Cheng, Yunyi Jia, Joyce Chai, and Ning Xi. 2014 · 2014
Earlier work this paper cites.
Asking for help using inverse semantics
Stefanie Tellex, Ross A. Knepper, Adrian Li, Daniela Rus, and Nicholas Roy. 2014 · 2014
Earlier work this paper cites.
Learning to search better than your teacher
Kai-Wei Chang, Akshay Krishnamurthy, Alekh Agarwal, Hal Daume III, and John Langford. 2015 · 2015
Earlier work this paper cites.

Similar

Effective approaches to attention-based neural machine translation
Minh-Thang Luong, Hieu Pham, and Christopher D Manning. 2015 · 2015
Cited alongside, same era.
Imagenet large scale visual recognition challenge
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. 2015 · 2015
Cited alongside, same era.
Openai gym
Original
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. 2016 · 2016
Cited alongside, same era.
Collaborative language grounding toward situated human-robot dialogue
Joyce Y Chai, Rui Fang, Changsong Liu, and Lanbo She. 2016 · 2016
Cited alongside, same era.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016 · 2016
Cited alongside, same era.
Vizdoom: A doom-based ai research platform for visual reinforcement learning
Michał Kempka, Marek Wydmuch, Grzegorz Runc, Jakub Toczek, and Wojciech Jaśkowski. 2016 · 2016
Cited alongside, same era.
Layer normalization
Original
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016 · 2016
Cited alongside, same era.
Jointly learning grounded task structures from language instruction and visual demonstration
Changsong Liu, Shaohua Yang, Sari Saba-Sadiya, Nishant Shukla, Yunzhong He, Song-Chun Zhu, and Joyce Chai. 2016 · 2016
Cited alongside, same era.
Matterport3D: Learning from RGB-D data in indoor environments
Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. 2017 · 2017
Cited alongside, same era.
Progressive growing of gans for improved quality, stability, and variation
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. 2017 · 2017
Cited alongside, same era.
Mapping instructions and visual observations to actions with reinforcement learning
Dipendra Misra, John Langford, and Yoav Artzi. 2017 · 2017
Cited alongside, same era.
Structured prediction via learning to search under bandit feedback
Amr Sharaf and Hal Daumé III. 2017 · 2017
Cited alongside, same era.

Then

Learning dexterous in-hand manipulation
Original
Marcin Andrychowicz, Bowen Baker, Maciek Chociej, Rafal Jozefowicz, Bob McGrew, Jakub Pachocki, Arthur Petron, Matthias Plappert, Glenn Powell, Alex Ray, et al. 2018 · 2018
Later among the works it cites.
Language to action: Towards interactive task learning with physical agents
Joyce Y Chai, Qiaozi Gao, Lanbo She, Shaohua Yang, Sari Saba-Sadiya, and Guangyue Xu. 2018 · 2018
Later among the works it cites.
Textworld: A learning environment for text-based games
Marc-Alexandre Côté, Ákos Kádár, Xingdi Yuan, Ben Kybartas, Tavian Barnes, Emery Fine, James Moore, Matthew Hausknecht, Layla El Asri, Mahmoud Adada, Wendy Tay, and Adam Trischler. 2018 · 2018
Later among the works it cites.
Embodied question answering
Abhishek Das, Samyak Datta, Georgia Gkioxari, Stefan Lee, Devi Parikh, and Dhruv Batra. 2018 · 2018
Later among the works it cites.
Speaker-follower models for vision-and-language navigation
Daniel Fried, Ronghang Hu, Volkan Cirik, Anna Rohrbach, Jacob Andreas, Louis-Philippe Morency, Taylor Berg-Kirkpatrick, Kate Saenko, Dan Klein, and Trevor Darrell. 2018 · 2018
Later among the works it cites.
Qt-opt: Scalable deep reinforcement learning for vision-based robotic manipulation
Dmitry Kalashnikov, Alex Irpan, Peter Pastor, Julian Ibarz, Alexander Herzog, Eric Jang, Deirdre Quillen, Ethan Holly, Mrinal Kalakrishnan, Vincent Vanhoucke, et al. 2018 · 2018
Later among the works it cites.
A style-based generator architecture for generative adversarial networks
Tero Karras, Samuli Laine, and Timo Aila. 2018 · 2018
Later among the works it cites.
Mapping instructions to actions in 3d environments with visual goal prediction
Dipendra Misra, Andrew Bennett, Valts Blukis, Eyvind Niklasson, Max Shatkhin, and Yoav Artzi. 2018 · 2018
Later among the works it cites.
Talk the walk: Navigating new york city through grounded dialogue
Original
Harm de Vries, Kurt Shuster, Dhruv Batra, Devi Parikh, Jason Weston, and Douwe Kiela. 2018 · 2018
Later among the works it cites.
Look before you leap: Bridging model-free and model-based reinforcement learning for planned-ahead vision-and-language navigation
Xin Wang, Wenhan Xiong, Hongmin Wang, and William Yang Wang. 2018 · 2018
Later among the works it cites.
Touchdown: Natural language navigation and spatial reasoning in visual street environments
Howard Chen, Alane Shur, Dipendra Misra, Noah Snavely, Ian Artzi, Yoav, Stephen Gould, and Anton van den Hengel. 2019 · 2019
Closest in time.
Babyai: A platform to study the sample efficiency of grounded language learning
Maxime Chevalier-Boisvert, Dzmitry Bahdanau, Salem Lahlou, Lucas Willems, Chitwan Saharia, Thien Huu Nguyen, and Yoshua Bengio. 2019 · 2019
Closest in time.
Vision-based navigation with language-based assistance via imitation learning with indirect intervention
Khanh Nguyen, Debadeepta Dey, Chris Brockett, and Bill Dolan. 2019 · 2019
Closest in time.
Reinforced cross-modal matching and self-supervised imitation learning for vision-language navigation
Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fang Wang, William Yang Wang, and Lei Zhang. 2019 · 2019
Closest in time.
Embodied question answering in photorealistic environments with point cloud perception
Erik Wijmans, Samyak Datta, Oleksandr Maksymets, Abhishek Das, Georgia Gkioxari, Stefan Lee, Irfan Essa, Devi Parikh, and Dhruv Batra. 2019 · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…