Fetching the paper…

Reading the bibliography…

2021

Rapid Exploration for Open-World Navigation with Latent Goal Models

Shah, Dhruv, Eysenbach, Benjamin, Kahn, Gregory et al.

Original
PDF
Post on X

Understand

We describe a robotic learning system for autonomous exploration and navigation in diverse, open-world environments.

At the core of our method is a learned latent variable model of distances and actions, along with a non-parametric topological memory of images.
We use an information bottleneck to regularize the learned policy, giving us (i) a compact visual representation of goals, (ii) improved generalization capabilities, and (iii) a mechanism for sampling feasible goals for exploration.
Trained on a large offline dataset of prior experience, the model acquires a representation of visual goals that is robust to task-irrelevant distractors.

Rapid Exploration for Open-World Navigation with Latent Goal Models · Around

Built on

B. Kuipers and Y.-T. Byun, “A robot exploration and mapping strategy based on a semantic hierarchy of spatial representations,” Robotics and Autonomous Systems , 1991, special Issue Toward Learning Robots
1991
Earlier work this paper cites.
L. P. Kaelbling, “Learning to achieve goals,” in IJCAI . Citeseer, 1993, pp. 1094–1099
1993
Earlier work this paper cites.
B. Yamauchi, “A frontier-based approach for autonomous exploration,” in IEEE International Symposium on Computational Intelligence in Robotics and Automation (CIRA) , 1997
1997
Earlier work this paper cites.
N. Tishby, F. C. Pereira, and W. Bialek, “The information bottleneck method,” arXiv preprint physics/0004057 , 2000
2000
Earlier work this paper cites.
F. Bourgault, A. A. Makarenko, S. B. Williams, B. Grocholsky, and H. F. Durrant-Whyte, “Information based adaptive robotic exploration,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , vol. 1, 2002, pp. 540–545 vol.1
2002
Earlier work this paper cites.
M. E. Taylor and P. Stone, “Cross-domain transfer for reinforcement learning,” in Proceedings of the 24th international conference on Machine learning , 2007, pp. 879–886
2007
Earlier work this paper cites.
T. Kollar and N. Roy, “Efficient optimization of information-theoretic exploration in slam,” in Proceedings of the AAAI Conference on Artificial Intelligence (AAAI) , 2008
2008
Earlier work this paper cites.
D. Holz, N. Basilico, F. Amigoni, and S. Behnke, “A comparative evaluation of exploration strategies and heuristics to improve them,” 2011
2011
Earlier work this paper cites.
A. Lazaric, “Transfer in reinforcement learning: a framework and a survey,” in Reinforcement Learning . Springer, 2012, pp. 143–173
2012
Earlier work this paper cites.
D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114 , 2013
Original
2013
Earlier work this paper cites.
B. Charrow, S. Liu, V. Kumar, and N. Michael, “Information-theoretic mapping using cauchy-schwarz quadratic mutual information,” in IEEE International Conference on Robotics and Automation (ICRA) , 2015
2015
Earlier work this paper cites.
B. C. Stadie, S. Levine, and P. Abbeel, “Incentivizing exploration in reinforcement learning with deep predictive models,” arXiv preprint arXiv:1507.00814 , 2015
Original
2015
Earlier work this paper cites.
E. Parisotto, J. L. Ba, and R. Salakhutdinov, “Actor-mimic: Deep multitask and transfer reinforcement learning,” arXiv preprint arXiv:1511.06342 , 2015
Original
2015
Earlier work this paper cites.
T. Schaul, D. Horgan, K. Gregor, and D. Silver, “Universal Value Function Approximators,” in International Conference on Machine Learning (ICML) , 2015
2015
Earlier work this paper cites.
R. Houthooft, X. Chen, Y. Duan, J. Schulman, F. De Turck, and P. Abbeel, “Vime: Variational information maximizing exploration,” arXiv preprint arXiv:1605.09674 , 2016
Original
2016
Earlier work this paper cites.
M. G. Bellemare, S. Srinivasan, G. Ostrovski, T. Schaul, D. Saxton, and R. Munos, “Unifying count-based exploration and intrinsic motivation,” arXiv preprint arXiv:1606.01868 , 2016
Original
2016
Earlier work this paper cites.
Y. Duan, J. Schulman, X. Chen, P. L. Bartlett, I. Sutskever, and P. Abbeel, “Rl 2 : Fast reinforcement learning via slow reinforcement learning,” arXiv preprint arXiv:1611.02779 , 2016
Original
2016
Earlier work this paper cites.
A. A. Alemi, I. Fischer, J. V. Dillon, and K. Murphy, “Deep variational information bottleneck,” arXiv preprint arXiv:1612.00410 , 2016
Original
2016
Earlier work this paper cites.
D. Pathak, P. Agrawal, A. A. Efros, and T. Darrell, “Curiosity-driven exploration by self-supervised prediction,” in International Conference on Machine Learning . PMLR, 2017, pp. 2778–2787
2017
Earlier work this paper cites.
A. Gupta, C. Devin, Y. Liu, P. Abbeel, and S. Levine, “Learning invariant feature spaces to transfer skills with reinforcement learning,” arXiv preprint arXiv:1703.02949 , 2017
Original
2017
Earlier work this paper cites.
J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal Policy Optimization Algorithms,” 2017
2017
Earlier work this paper cites.

Similar

A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications,” 2017
2017
Cited alongside, same era.
N. Savinov, A. Dosovitskiy, and V. Koltun, “Semi-Parametric Topological Memory for Navigation,” in International Conference on Learning Representations , 2018
2018
Cited alongside, same era.
A. Faust, K. Oslund, O. Ramirez, A. Francis, L. Tapia, M. Fiser, and J. Davidson, “Prm-rl: Long-range robotic navigation tasks by combining reinforcement learning and sampling-based planning,” in 2018 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2018, pp. 5113–5120
2018
Cited alongside, same era.
R. Bajcsy, Y. Aloimonos, and J. K. Tsotsos, “Revisiting active perception,” Autonomous Robots , vol. 42, no. 2, pp. 177–196, 2018
2018
Cited alongside, same era.

Then

H. Sun, Z. Li, X. Liu, D. Lin, and B. Zhou, “Policy continuation with hindsight inverse dynamics,” arXiv preprint arXiv:1910.14055 , 2019
Original
2019
Later among the works it cites.
R. K. Srivastava, P. Shyam, F. Mutz, W. Jaśkowski, and J. Schmidhuber, “Training agents using upside-down reinforcement learning,” arXiv preprint arXiv:1912.02877 , 2019
Original
2019
Later among the works it cites.
V. H. Pong, M. Dalal, S. Lin, A. Nair, S. Bahl, and S. Levine, “Skew-fit: State-covering self-supervised reinforcement learning,” arXiv preprint arXiv:1903.03698 , 2019
Original
2019
Later among the works it cites.
C. Colas, P. Fournier, M. Chetouani, O. Sigaud, and P.-Y. Oudeyer, “Curious: intrinsically motivated modular multi-goal reinforcement learning,” in International conference on machine learning . PMLR, 2019, pp. 1331–1340
2019
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Y. Burda, H. Edwards, A. Storkey, and O. Klimov, “Exploration by random network distillation,” arXiv preprint arXiv:1810.12894 , 2018

2018

Cited alongside, same era.

A. Mirchev, B. Kayalibay, M. Soelch, P. van der Smagt, and J. Bayer, “Approximate bayesian inference in spatial environments,” arXiv preprint arXiv:1805.07206 , 2018

2018

Cited alongside, same era.

S. Sæmundsson, K. Hofmann, and M. P. Deisenroth, “Meta reinforcement learning with latent variable gaussian processes,” arXiv preprint arXiv:1803.07551 , 2018

2018

Cited alongside, same era.

A. Nagabandi, I. Clavera, S. Liu, R. S. Fearing, P. Abbeel, S. Levine, and C. Finn, “Learning to adapt in dynamic, real-world environments through meta-reinforcement learning,” arXiv preprint arXiv:1803.11347 , 2018

2018

Cited alongside, same era.

V. Pong, S. Gu, M. Dalal, and S. Levine, “Temporal difference models: Model-free deep rl for model-based control,” arXiv preprint arXiv:1802.09081 , 2018

2018

Cited alongside, same era.

A. Achille and S. Soatto, “Emergence of invariance and disentanglement in deep representations,” The Journal of Machine Learning Research , vol. 19, no. 1, pp. 1947–1980, 2018

2018

Cited alongside, same era.

W. Tabib, K. Goel, J. Yao, M. Dabhi, C. Boirum, and N. Michael, “Real-time information-theoretic exploration with gaussian mixture model maps.” in Robotics: Science and Systems , 2019

2019

Cited alongside, same era.

B. Eysenbach, R. R. Salakhutdinov, and S. Levine, “Search on the Replay Buffer: Bridging Planning and RL,” in Advances in Neural Information Processing Systems (NeurIPS) , 2019

2019

Cited alongside, same era.

X. Meng, N. Ratliff, Y. Xiang, and D. Fox, “Scaling Local Control to Large-Scale Topological Navigation,” in IEEE International Conference on Robotics and Automation (ICRA) , 2020

2020

Later among the works it cites.

C. Lynch, M. Khansari, T. Xiao, V. Kumar, J. Tompson, S. Levine, and P. Sermanet, “Learning latent plans from play,” in Conference on Robot Learning . PMLR, 2020, pp. 1113–1132

2020

Later among the works it cites.

Y. Satsangi, S. Lim, S. Whiteson, F. Oliehoek, and M. White, “Maximizing information gain in partially observable environments via prediction reward,” arXiv preprint arXiv:2005.04912 , 2020

2020

Later among the works it cites.

D. S. Chaplot, D. Gandhi, S. Gupta, A. Gupta, and R. Salakhutdinov, “Learning to Explore using Active Neural SLAM,” in International Conference on Learning Representations (ICLR) , 2020

2020

Later among the works it cites.

D. Singh Chaplot, R. Salakhutdinov, A. Gupta, and S. Gupta, “Neural topological slam for visual navigation,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2020

2020

Later among the works it cites.

D. S. Chaplot, H. Jiang, S. Gupta, and A. Gupta, “Semantic curiosity for active visual learning,” in ECCV , 2020

2020

Later among the works it cites.

B. Eysenbach, R. Salakhutdinov, and S. Levine, “C-learning: Learning to achieve goals via recursive classification,” arXiv preprint arXiv:2011.08909 , 2020

2020

Later among the works it cites.

N. Rhinehart, R. McAllister, and S. Levine, “Deep imitative models for flexible inference, planning, and control,” in International Conference on Learning Representations , 2020

2020

Later among the works it cites.

L. Zhang, G. Yang, and B. C. Stadie, “World model as a graph: Learning latent landmarks for planning,” arXiv preprint arXiv:2011.12491 , 2020

2020

Later among the works it cites.

S. Pitis, H. Chan, S. Zhao, B. Stadie, and J. Ba, “Maximum entropy gain exploration for long horizon multi-goal reinforcement learning,” in International Conference on Machine Learning . PMLR, 2020, pp. 7750–7761

2020

Later among the works it cites.

K. Liu, T. Kurutach, C. Tung, P. Abbeel, and A. Tamar, “Hallucinative topological memory for zero-shot visual planning,” in International Conference on Machine Learning . PMLR, 2020, pp. 6259–6270

2020

Later among the works it cites.

G. Kahn, P. Abbeel, and S. Levine, “BADGR: An Autonomous Self-Supervised Learning-Based Navigation System,” 2020

2020

Later among the works it cites.

E. Wijmans, A. Kadian, A. Morcos, S. Lee, I. Essa, D. Parikh, M. Savva, and D. Batra, “DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames,” in International Conference on Learning Representations (ICLR) , 2020

2020

Later among the works it cites.

D. Shah, B. Eysenbach, G. Kahn, N. Rhinehart, and S. Levine, “ViNG: Learning Open-World Navigation with Visual Goals,” in IEEE International Conference on Robotics and Automation (ICRA) , 2021

2021

Closest in time.

N. Yokoyama, S. Ha, and D. Batra, “Success weighted by completion time: A dynamics-aware evaluation criteria for embodied navigation,” 2021

2021

Closest in time.

S. Gamrian and Y. Goldberg, “Transfer learning for related reinforcement learning tasks via image-to-image translation,” in International Conference on Machine Learning . PMLR, 2019, pp. 2063–2072

2072

Closest in time.