Fetching the paper…

Reading the bibliography…

2021

Adversarial Environment Generation for Learning to Navigate the Web

Gur, Izzeddin, Jaques, Natasha, Malta, Kevin et al.

Original
PDF
Post on X

Understand

Learning to autonomously navigate the web is a difficult sequential decision making task.

The state and action spaces are large and combinatorial in nature, and websites are dynamic environments consisting of several pages.
One of the bottlenecks of training web navigation agents is providing a learnable curriculum of training environments that can cover the large variety of real-world websites.
Therefore, we propose using Adversarial Environment Generation (AEG) to generate challenging web environments in which to train reinforcement learning (RL) agents.

Adversarial Environment Generation for Learning to Navigate the Web · Around

Built on

On finding local nash equilibria (and only local nash equilibria) in zero-sum games
Original
Eric V Mazumdar, Michael I Jordan, and S Shankar Sastry · 1901
Earlier work this paper cites.
Policy-gradient algorithms have no guarantees of convergence in continuous action and state multi-agent settings
Original
Eric Mazumdar, Lillian J Ratliff, Michael I Jordan, and S Shankar Sastry · 1907
Earlier work this paper cites.
Evolutionary robotics and the radical envelope-of-noise hypothesis
Nick Jakobi · 1997
Earlier work this paper cites.
Cad2rl: Real single-image flight without a single real image
Original
Fereshteh Sadeghi and Sergey Levine · 2016
Earlier work this paper cites.
Automated curriculum learning for neural networks
Original
Alex Graves, Marc G Bellemare, Jacob Menick, Remi Munos, and Koray Kavukcuoglu · 2017
Earlier work this paper cites.
World of bits: An open-domain platform for web-based agents
Tianlin Shi, Andrej Karpathy, Linxi Fan, Jonathan Hernandez, and Percy Liang · 2017
Earlier work this paper cites.

Similar

Intrinsic motivation and automatic curricula via asymmetric self-play
Original
Sainbayar Sukhbaatar, Zeming Lin, Ilya Kostrikov, Gabriel Synnaeve, Arthur Szlam, and Rob Fergus · 2017
Cited alongside, same era.
Domain randomization for transferring deep neural networks from simulation to the real world
Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, and Pieter Abbeel · 2017
Cited alongside, same era.
Reinforcement learning on web interfaces using workflow-guided exploration
Original
Evan Zheran Liu, Kelvin Guu, Panupong Pasupat, Tianlin Shi, and Percy Liang · 2018
Cited alongside, same era.
Learning to navigate the web
Izzeddin Gur, Uli Rueckert, Aleksandra Faust, and Dilek Hakkani-Tur · 2019
Cited alongside, same era.
Autocurricula and the emergence of innovation from social interaction: A manifesto for multi-agent intelligence research
Original

Then

Paired open-ended trailblazer (poet): Endlessly generating increasingly complex and diverse learning environments and their solutions
Original
Rui Wang, Joel Lehman, Jeff Clune, and Kenneth O Stanley · 2019
Later among the works it cites.
Learning with amigo: Adversarially motivated intrinsic goals
Original
Andres Campero, Roberta Raileanu, Heinrich Küttler, Joshua B Tenenbaum, Tim Rocktäschel, and Edward Grefenstette · 2020
Later among the works it cites.
Emergent complexity and zero-shot transfer via unsupervised environment design
Michael Dennis, Natasha Jaques, Eugene Vinitsky, Alexandre Bayen, Stuart Russell, Andrew Critch, and Sergey Levine · 2020
Later among the works it cites.
Teacher algorithms for curriculum learning of deep rl in continuously parameterized environments
Rémy Portelas, Cédric Colas, Katja Hofmann, and Pierre-Yves Oudeyer · 2020
Later among the works it cites.
Enhanced poet: Open-ended reinforcement learning through unbounded invention of learning challenges and their solutions

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Joel Z Leibo, Edward Hughes, Marc Lanctot, and Thore Graepel · 2019

Cited alongside, same era.

Teacher-student curriculum learning

Tambet Matiisen, Avital Oliver, Taco Cohen, and John Schulman · 2019

Cited alongside, same era.

Rui Wang, Joel Lehman, Aditya Rawal, Jiale Zhi, Yulun Li, Jeff Clune, and Kenneth O Stanley · 2020

Later among the works it cites.