MMInA: Benchmarking Multihop Multimodal Internet Agents

Modeling adaptive autonomous agents

Pattie Maes · 1993

Earlier work this paper cites.

Adaptive behavior in autonomous agents

Original

Tom Ziemke · 1998

Earlier work this paper cites.

Autonomous artificial intelligent agents

Razvan V Florian · 2003

Earlier work this paper cites.

Erica: Interaction mining mobile apps

Biplab Deka, Zifeng Huang, and Ranjitha Kumar · 2016

Earlier work this paper cites.

World of bits: An open-domain platform for web-based agents

Tianlin Shi, Andrej Karpathy, Linxi Fan, Jonathan Hernandez, and Percy Liang · 2017

Earlier work this paper cites.

Reinforcement learning on web interfaces using workflow-guided exploration

Original

Evan Zheran Liu, Kelvin Guu, Panupong Pasupat, Tianlin Shi, and Percy Liang · 2018

Earlier work this paper cites.

Virtualhome: Simulating household activities via programs

Xavier Puig, Kevin Ra, Marko Boben, Jiaman Li, Tingwu Wang, Sanja Fidler, and Antonio Torralba · 2018

Earlier work this paper cites.

The artificial life route to artificial intelligence: Building embodied, situated agents

Luc Steels and Rodney Brooks · 2018

Earlier work this paper cites.

Learning transferable visual models from natural language supervision

Original

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al · 2021

Earlier work this paper cites.

Zero-shot text-to-image generation

Original

Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever · 2021

Earlier work this paper cites.

Androidenv: A reinforcement learning platform for android

Original

Daniel Toyama, Philippe Hamel, Anita Gergely, Gheorghe Comanici, Amelia Glaese, Zafarali Ahmed, Tyler Jackson, Shibl Mourad, and Doina Precup · 2021

Earlier work this paper cites.

Finetuned language models are zero-shot learners

Original

Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le · 2021

Earlier work this paper cites.

Grounding open-domain instructions to automate web support tasks

Original

Nancy Xu, Sam Masling, Michael Du, Giovanni Campagna, Larry Heck, James Landay, and Monica S Lam · 2021

Earlier work this paper cites.

Flamingo: a visual language model for few-shot learning

Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al · 2022

Earlier work this paper cites.

Scaling instruction-finetuned language models

Original

Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al · 2022

Earlier work this paper cites.

Training language models to follow instructions with human feedback, 2022

Original

Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al · 2022

Earlier work this paper cites.

High-resolution image synthesis with latent diffusion models

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer · 2022

Earlier work this paper cites.

Laion-5b: An open large-scale dataset for training next generation image-text models

Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al · 2022

Cited alongside, same era.

Webshop: Towards scalable real-world web interaction with grounded language agents

Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan · 2022

Cited alongside, same era.

React: Synergizing reasoning and acting in language models

Original

Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao · 2022

Cited alongside, same era.

Introducing our multimodal models, 2023

Rohan Bavishi, Erich Elsen, Curtis Hawthorne, Maxwell Nye, Augustus Odena, Arushi Somani, and Sağnak Taşırlar · 2023

Cited alongside, same era.

Mind2web: Towards a generalist agent for the web

Original

Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su · 2023

Code llama: Open foundation models for code

Original

Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, et al · 2023

Later among the works it cites.

Language modelling with pixels, 2023

Phillip Rust, Jonas F. Lotz, Emanuele Bugliarello, Elizabeth Salesky, Miryam de Lhoneux, and Desmond Elliott · 2023

Later among the works it cites.

Appagent: Multimodal agents as smartphone users

Original

Zhao Yang, Jiaxuan Liu, Yucheng Han, Xin Chen, Zebiao Huang, Bin Fu, and Gang Yu · 2023

Later among the works it cites.

Chatbridge: Bridging modalities with large language model as a language catalyst

Original

Zijia Zhao, Longteng Guo, Tongtian Yue, Sihan Chen, Shuai Shao, Xinxin Zhu, Zehuan Yuan, and Jing Liu · 2023

Later among the works it cites.

Webarena: A realistic web environment for building autonomous agents