Fetching the paper…
Reading the bibliography…
Nowadays, research on GUI agents is a hot topic in the AI community.
Classifying and qualifying gui defects
Valéria Lelli, Arnaud Blouin, and Benoit Baudry · 2015
Earlier work this paper cites.
World of bits: An open-domain platform for web-based agents
Tianlin Shi, Andrej Karpathy, Linxi Fan, Jonathan Hernandez, and Percy Liang · 2017
Earlier work this paper cites.
Continuous integration and visual gui testing: Benefits and drawbacks in industrial practice
Emil Alégroth, Arvid Karlsson, and Alexander Radway · 2018
Earlier work this paper cites.
Izzeddin Gur, Ulrich Rueckert, Aleksandra Faust, and Dilek Hakkani-Tur · 2018
Earlier work this paper cites.
Appflow: using machine learning to synthesize robust, reusable ui tests
Gang Hu, Linjie Zhu, and Junfeng Yang · 2018
Earlier work this paper cites.
Mutapk: Source-codeless mutant generation for android apps
Camilo Escobar-Velásquez, Michael Osorio-Riaño, and Mario Linares-Vásquez · 2019
Earlier work this paper cites.
Mutapk 2.0: A tool for reducing mutation testing effort of android apps
Camilo Escobar-Velásquez, Diego Riveros, and Mario Linares-Vásquez · 2020
Earlier work this paper cites.
Mapping natural language instructions to mobile ui action sequences
Yang Li, Jiacong He, Xin Zhou, Yuan Zhang, and Jason Baldridge · 2020
Earlier work this paper cites.
The power of scale for parameter-efficient prompt tuning
Brian Lester, Rami Al-Rfou, and Noah Constant · 2021
Earlier work this paper cites.
Benchmarking automated gui testing for android against real-world bugs
Ting Su, Jue Wang, and Zhendong Su · 2021
Earlier work this paper cites.
Flamingo: a visual language model for few-shot learning
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al · 2022
Earlier work this paper cites.
Spotlight: Mobile ui understanding using vision-language models with a focus
Gang Li and Yang Li · 2022
Earlier work this paper cites.
Training language models to follow instructions with human feedback
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al · 2022
Earlier work this paper cites.
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al · 2023
Earlier work this paper cites.
Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou · 2023
Cited alongside, same era.
Palm: Scaling language modeling with pathways
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al · 2023
Cited alongside, same era.
Fill in the blank: Context-aware automated text input generation for mobile gui testing
Zhe Liu, Chunyang Chen, Junjie Wang, Xing Che, Yuekai Huang, Jun Hu, and Qing Wang · 2023
Cited alongside, same era.
Droidbot-gpt: A lightweight model-driven tool for automated gui testing on android
MobileLLM · 2023
Cited alongside, same era.
Llama: Open and efficient foundation language models
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al · 2023
Windows agent arena: Evaluating multi-modal os agents at scale
Rogerio Bonatti, Dan Zhao, Francesco Bonacci, Dillon Dupont, Sara Abdali, Yinheng Li, Yadong Lu, Justin Wagle, Kazuhito Koishida, Arthur Bucker, et al · 2024
Closest in time.
Comprehensive semantic repair of obsolete gui test scripts for mobile applications
Shaoheng Cao, Minxue Pan, Yu Pei, Wenhua Yang, Tian Zhang, Linzhang Wang, and Xuandong Li · 2024
Closest in time.
Seeclick: Harnessing gui grounding for advanced visual gui agents
Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Yantao Li, Jianbing Zhang, and Zhiyong Wu · 2024
Closest in time.
Mind2web: Towards a generalist agent for the web
Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Sam Stevens, Boshi Wang, Huan Sun, and Yu Su · 2024
Closest in time.
Github repository of applications, 2024
GitHub · 2024
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Empowering llm to use smartphone for intelligent task automation
Hao Wen, Yuanchun Li, Guohong Liu, Shanhui Zhao, Tao Yu, Toby Jia-Jun Li, Shiqi Jiang, Yunhao Liu, Yaqin Zhang, and Yunxin Liu · 2023
Cited alongside, same era.
An empirical study of functional bugs in android apps
Yiheng Xiong, Mengqian Xu, Ting Su, Jingling Sun, Jue Wang, He Wen, Geguang Pu, Jifeng He, and Zhendong Su · 2023
Cited alongside, same era.
Appagent: Multimodal agents as smartphone users
Chi Zhang, Zhao Yang, Jiaxuan Liu, Yucheng Han, Xin Chen, Zebiao Huang, Bin Fu, and Gang Yu · 2023
Cited alongside, same era.
Synapse: Trajectory-as-exemplar prompting with memory for computer control
Longtao Zheng, Rundong Wang, Xinrun Wang, and Bo An · 2023
Cited alongside, same era.
Webarena: A realistic web environment for building autonomous agents
Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, et al · 2023
Cited alongside, same era.
Accessed: 2024-11-13
F-droid: Free and open source android app repository, 2024 · 2024
Cited alongside, same era.
Accessed: 2024-11-13
Google play store, 2024 · 2024
Cited alongside, same era.
Cogagent: A visual language model for gui agents
Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxiao Dong, Ming Ding, et al · 2024
Closest in time.
Auitestagent: Automatic requirements oriented gui function testing
Yongxiang Hu, Xuan Wang, Yingchuan Wang, Yu Zhang, Shiyu Guo, Chaoyi Chen, Xin Wang, and Yangfan Zhou · 2024
Closest in time.
Comprehensive cognitive llm agent for smartphone gui automation
Xinbei Ma, Zhuosheng Zhang, and Hai Zhao · 2024
Closest in time.
Agent q: Advanced reasoning and learning for autonomous ai agents
Pranav Putta, Edmund Mills, Naman Garg, Sumeet Motwani, Chelsea Finn, Divyansh Garg, and Rafael Rafailov · 2024
Closest in time.
Androidinthewild: A large-scale dataset for android device control
Christopher Rawles, Alice Li, Daniel Rodriguez, Oriana Riva, and Timothy Lillicrap · 2024
Closest in time.
Ferret-ui: Grounded mobile ui understanding with multimodal llms
Keen You, Haotian Zhang, Eldon Schoop, Floris Weers, Amanda Swearngin, Jeffrey Nichols, Yinfei Yang, and Zhe Gan · 2024
Closest in time.
Android in the zoo: Chain-of-action-thought for gui agents
Jiwen Zhang, Jihao Wu, Yihua Teng, Minghui Liao, Nuo Xu, Xiao Xiao, Zhongyu Wei, and Duyu Tang · 2024
Closest in time.
Ferret-ui: Grounded mobile ui understanding with multimodal llms
Keen You, Haotian Zhang, Eldon Schoop, Floris Weers, Amanda Swearngin, Jeffrey Nichols, Yinfei Yang, and Zhe Gan · 2025
Closest in time.