CHOP: Mobile Operating Assistant with Constrained High-frequency Optimized Subtask Planning

Language models are few-shot learners

Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020 · 1901

Earlier work this paper cites.

Towards completeness-oriented tool retrieval for large language models

Changle Qu, Sunhao Dai, Xiaochi Wei, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Jun Xu, and Ji-Rong Wen. 2024 · 1940

Earlier work this paper cites.

Bleu: a method for automatic evaluation of machine translation

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002 · 2002

Earlier work this paper cites.

Rouge: A package for automatic evaluation of summaries

Chin-Yew Lin. 2004 · 2004

Earlier work this paper cites.

Sugilite: creating multimodal smartphone automation by demonstration

Toby Jia-Jun Li, Amos Azaria, and Brad A Myers. 2017 · 2017

Earlier work this paper cites.

Pumice: A multi-modal agent that learns concepts and conditionals from natural language and demonstrations

Toby Jia-Jun Li, Marissa Radensky, Justin Jia, Kirielle Singarajah, Tom M Mitchell, and Brad A Myers. 2019 · 2019

Earlier work this paper cites.

Uibert: Learning generic multimodal representations for ui understanding

Original

Chongyang Bai, Xiaoxue Zang, Ying Xu, Srinivas Sunkara, Abhinav Rastogi, Jindong Chen, et al. 2021 · 2021

Earlier work this paper cites.

Actionbert: Leveraging user actions for semantic understanding of user interfaces

Zecheng He, Srinivas Sunkara, Xiaoxue Zang, Ying Xu, Lijuan Liu, Nevan Wichers, Gabriel Schubiner, Ruby Lee, and Jindong Chen. 2021 · 2021

Earlier work this paper cites.

Screen2vec: Semantic embedding of gui screens and gui components

Toby Jia-Jun Li, Lindsay Popowski, Tom Mitchell, and Brad A Myers. 2021 · 2021

Earlier work this paper cites.

Screen2words: Automatic mobile ui summarization with multimodal learning

Bryan Wang, Gang Li, Xin Zhou, Zhourong Chen, Tovi Grossman, and Yang Li. 2021 · 2021

Earlier work this paper cites.

Do as i can, not as i say: Grounding language in robotic affordances

Original

Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, et al. 2022 · 2022

Earlier work this paper cites.

Meta-gui: Towards multi-modal conversational agents on mobile gui

Original

Liangtai Sun, Xingyu Chen, Lu Chen, Tianle Dai, Zichen Zhu, and Kai Yu. 2022 · 2022

Earlier work this paper cites.

Humans decompose tasks by trading off utility and computational cost

Carlos G Correa, Mark K Ho, Frederick Callaway, Nathaniel D Daw, and Thomas L Griffiths. 2023 · 2023

Cited alongside, same era.

Spotlight: Mobile ui understanding using vision-language models with a focus

Gang Li and Yang Li. 2023 · 2023

Cited alongside, same era.

Camel: communicative agents for" mind" exploration of large language model society

Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023 · 2023

Cited alongside, same era.

Generative agents: Interactive simulacra of human behavior

Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023 · 2023

Cited alongside, same era.

Appagent: Multimodal agents as smartphone users

Original

Chi Zhang, Zhao Yang, Jiaxuan Liu, Yucheng Han, Xin Chen, Zebiao Huang, Bin Fu, and Gang Yu. 2023 · 2023

Cited alongside, same era.

Autowebglm: Bootstrap and reinforce a large language model-based web navigating agent

Original

Hanyu Lai, Xiao Liu, Iat Long Iong, Shuntian Yao, Yuxuan Chen, Pengbo Shen, Hao Yu, Hanchen Zhang, Xiaohan Zhang, Yuxiao Dong, et al. 2024 · 2024

Later among the works it cites.

Mobilegpt: Augmenting llm with human-like app memory for mobile task automation

Sunjae Lee, Junyoung Choi, Jungjae Lee, Munim Hasan Wasi, Hojun Choi, Steve Ko, Sangeun Oh, and Insik Shin. 2024 · 2024

Later among the works it cites.

Showui: One vision-language-action model for generalist gui agent

Kevin Qinghong Lin, Linjie Li, Difei Gao, Zhengyuan Yang, Zechen Bai, Weixian Lei, Lijuan Wang, and Mike Zheng Shou. 2024 · 2024

Later among the works it cites.

Vision-driven automated mobile gui testing via multimodal large language model

Original

Zhe Liu, Cheng Li, Chunyang Chen, Junjie Wang, Boyu Wu, Yawen Wang, Jun Hu, and Qing Wang. 2024 · 2024

Later among the works it cites.

Gui agents: A survey