Fetching the paper…
Reading the bibliography…
State-of-the-art multimodal web agents, powered by Multimodal Large Language Models (MLLMs), can autonomously execute many web tasks by processing user instructions and interacting with graphical user interfaces (GUIs).
Evolutionary principles in self-referential learning
Jurgen Schmidhuber · 1987
Earlier work this paper cites.
Alvinn: An autonomous land vehicle in a neural network
Dean A Pomerleau · 1988
Earlier work this paper cites.
Learning from demonstration
Stefan Schaal · 1996
Earlier work this paper cites.
Algorithms for inverse reinforcement learning
Andrew Y Ng, Stuart Russell, et al · 2000
Earlier work this paper cites.
Robots that imitate humans
Cynthia Breazeal and Brian Scassellati · 2002
Earlier work this paper cites.
Apprenticeship learning via inverse reinforcement learning
Pieter Abbeel and Andrew Y Ng · 2004
Earlier work this paper cites.
Perception, cognition, and action in teams of robots
Manuela Veloso · 2005
Earlier work this paper cites.
Dynamic movement primitives-a framework for motor control in humans and humanoid robotics
Stefan Schaal · 2006
Earlier work this paper cites.
On learning, representing, and generalizing a task in a humanoid robot
Sylvain Calinon, Florent Guenter, and Aude Billard · 2007
Earlier work this paper cites.
Interactive robot task training through dialog and demonstration
Paul E Rybski, Kevin Yoon, Jeremy Stolarz, and Manuela M Veloso · 2007
Earlier work this paper cites.
Maximum entropy inverse reinforcement learning
Brian D Ziebart, Andrew L Maas, J Andrew Bagnell, Anind K Dey, et al · 2008
Earlier work this paper cites.
A survey of robot learning from demonstration
Brenna D Argall, Sonia Chernova, Manuela Veloso, and Brett Browning · 2009
Earlier work this paper cites.
Learning and reproduction of gestures by imitation
Sylvain Calinon, Florent D’halluin, Eric L Sauser, Darwin G Caldwell, and Aude G Billard · 2010
Earlier work this paper cites.
A reduction of imitation learning and structured prediction to no-regret online learning
Stéphane Ross, Geoffrey Gordon, and Drew Bagnell · 2011
Earlier work this paper cites.
Explaining and harnessing adversarial examples
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy · 2014
Earlier work this paper cites.
Learning driving styles for autonomous vehicles from demonstration
Markus Kuderer, Shilpa Gulati, and Wolfram Burgard · 2015
Earlier work this paper cites.
Generative adversarial imitation learning
Jonathan Ho and Stefano Ermon · 2016
Earlier work this paper cites.
Mastering the game of go with deep neural networks and tree search
David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al · 2016
Earlier work this paper cites.
Inverse kkt: Learning cost functions of manipulation tasks from demonstrations
Peter Englert, Ngo Anh Vien, and Marc Toussaint · 2017
Earlier work this paper cites.
Model-agnostic meta-learning for fast adaptation of deep networks
Chelsea Finn, Pieter Abbeel, and Sergey Levine · 2017
Earlier work this paper cites.
One-shot visual imitation learning via meta-learning
Chelsea Finn, Tianhe Yu, Tianhao Zhang, Pieter Abbeel, and Sergey Levine · 2017
Earlier work this paper cites.
Imitation learning: A survey of learning methods
Ahmed Hussein, Mohamed Medhat Gaber, Eyad Elyan, and Chrisina Jayne · 2017
Cited alongside, same era.
World of bits: An open-domain platform for web-based agents
Tianlin Shi, Andrej Karpathy, Linxi Fan, Jonathan Hernandez, and Percy Liang · 2017
Cited alongside, same era.
Reinforcement learning on web interfaces using workflow-guided exploration
Evan Zheran Liu, Kelvin Guu, Panupong Pasupat, Tianlin Shi, and Percy Liang · 2018
Cited alongside, same era.
On first-order meta-learning algorithms
A Nichol · 2018
Cited alongside, same era.
Extrapolating beyond suboptimal demonstrations via inverse reinforcement learning from observations
Daniel Brown, Wonjoon Goo, Prabhat Nagarajan, and Scott Niekum · 2019
Cited alongside, same era.
Experimental evidence on the productivity effects of generative artificial intelligence
Shakked Noy and Whitney Zhang · 2023
Later among the works it cites.
Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr · 2023
Later among the works it cites.
From pixels to ui actions: Learning to follow instructions via graphical user interfaces, 2023
Peter Shaw, Mandar Joshi, James Cohan, Jonathan Berant, Panupong Pasupat, Hexiang Hu, Urvashi Khandelwal, Kenton Lee, and Kristina Toutanova · 2023
Later among the works it cites.
Cogvlm: Visual expert for pretrained language models
Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Xixuan Song, et al · 2023
Later among the works it cites.
Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Tom B Brown · 2020
Cited alongside, same era.
Recent advances in robot learning from demonstration
Harish Ravichandar, Athanasios S Polydoros, Sonia Chernova, and Aude Billard · 2020
Cited alongside, same era.
Transformers: State-of-the-art natural language processing
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al · 2020
Cited alongside, same era.
Learning from suboptimal demonstration via self-supervised reward regression
Letian Chen, Rohan Paleja, and Matthew Gombolay · 2021
Cited alongside, same era.
Model-based inverse reinforcement learning from visual demonstrations
Neha Das, Sarah Bechtle, Todor Davchev, Dinesh Jayaraman, Akshara Rai, and Franziska Meier · 2021
Cited alongside, same era.
Generalization guarantees for imitation learning
Allen Ren, Sushant Veer, and Anirudha Majumdar · 2021
Cited alongside, same era.
A data-driven approach for learning to control computers
Peter C Humphreys, David Raposo, Tobias Pohlen, Gregory Thornton, Rachita Chhaparia, Alistair Muldal, Josh Abramson, Petko Georgiev, Adam Santoro, and Timothy Lillicrap · 2022
Cited alongside, same era.
Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao · 2023
Later among the works it cites.
Group preference optimization: Few-shot alignment of large language models
Siyan Zhao, John Dang, and Aditya Grover · 2023
Later among the works it cites.
In-context learning with long-context models: An in-depth exploration
Amanda Bertsch, Maor Ivgi, Uri Alon, Jonathan Berant, Matthew R Gormley, and Graham Neubig · 2024
Closest in time.
Workarena++: Towards compositional planning and reasoning-based common knowledge work tasks
Léo Boisvert, Megh Thakkar, Maxime Gasse, Massimo Caccia, De Chezelles, Thibault Le Sellier, Quentin Cappart, Nicolas Chapados, Alexandre Lacoste, and Alexandre Drouin · 2024
Closest in time.
Seeclick: Harnessing gui grounding for advanced visual gui agents
Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Yantao Li, Jianbing Zhang, and Zhiyong Wu · 2024
Closest in time.
A real-world webagent with planning, long context understanding, and program synthesis, 2024
Izzeddin Gur, Hiroki Furuta, Austin Huang, Mustafa Safdari, Yutaka Matsuo, Douglas Eck, and Aleksandra Faust · 2024
Closest in time.
Webvoyager: Building an end-to-end web agent with large multimodal models
Hongliang He, Wenlin Yao, Kaixin Ma, Wenhao Yu, Yong Dai, Hongming Zhang, Zhenzhong Lan, and Dong Yu · 2024
Closest in time.
Visualwebarena: Evaluating multimodal agents on realistic visual web tasks
Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Chong Lim, Po-Yu Huang, Graham Neubig, Shuyan Zhou, Ruslan Salakhutdinov, and Daniel Fried · 2024
Closest in time.
Autowebglm: Bootstrap and reinforce a large language model-based web navigating agent
Hanyu Lai, Xiao Liu, Iat Long Iong, Shuntian Yao, Yuxuan Chen, Pengbo Shen, Hao Yu, Hanchen Zhang, Xiaohan Zhang, Yuxiao Dong, et al · 2024
Closest in time.
What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning, 2024
Wei Liu, Weihao Zeng, Keqing He, Yong Jiang, and Junxian He · 2024
Closest in time.
Oracle AI agents help organizations achieve new levels of productivity, Sep 2024
Oracle · 2024
Closest in time.
Synatra: Turning indirect knowledge into direct demonstrations for digital agents at scale
Tianyue Ou, Frank F Xu, Aman Madaan, Jiarui Liu, Robert Lo, Abishek Sridhar, Sudipta Sengupta, Dan Roth, Graham Neubig, and Shuyan Zhou · 2024
Closest in time.
Direct preference optimization: Your language model is secretly a reward model
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn · 2024
Closest in time.
Your transformer is secretly linear
Anton Razzhigaev, Matvey Mikhalchuk, Elizaveta Goncharova, Nikolai Gerasimenko, Ivan Oseledets, Denis Dimitrov, and Andrey Kuznetsov · 2024
Closest in time.
Show, don’t tell: Aligning language models with demonstrated feedback
Omar Shaikh, Michelle Lam, Joey Hejna, Yijia Shao, Michael Bernstein, and Diyi Yang · 2024
Closest in time.
Beyond browsing: Api-based web agents
Yueqi Song, Frank Xu, Shuyan Zhou, and Graham Neubig · 2024
Closest in time.
Gpt-4v(ision) is a generalist web agent, if grounded
Boyuan Zheng, Boyu Gou, Jihyung Kil, Huan Sun, and Yu Su · 2024
Closest in time.