Improved GUI Grounding via Iterative Narrowing

Iterative visual reasoning beyond convolutions, 2018

Original

Xinlei Chen, Li-Jia Li, Li Fei-Fei, and Abhinav Gupta · 2018

Earlier work this paper cites.

Iterative shrinking for referring expression grounding using deep reinforcement learning, 2021

Original

Mingjie Sun, Jimin Xiao, and Eng Gee Lim · 2021

Earlier work this paper cites.

Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks

Original

Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, Bin Li, Ping Luo, Tong Lu, Yu Qiao, and Jifeng Dai · 2023

Earlier work this paper cites.

Cogagent: A visual language model for gui agents, 2023

Original

Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxuan Zhang, Juanzi Li, Bin Xu, Yuxiao Dong, Ming Ding, and Jie Tang · 2023

Earlier work this paper cites.

V*: Guided visual search as a core mechanism in multimodal llms, 2023

Original

Penghao Wu and Saining Xie · 2023

Earlier work this paper cites.

How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites

Original

Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al · 2024

Cited alongside, same era.

SeeClick: Harnessing GUI grounding for advanced visual GUI agents

Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Li YanTao, Jianbing Zhang, and Zhiyong Wu · 2024

Cited alongside, same era.

Navigating the digital world as humans do: Universal visual grounding for gui agents

Original

Boyu Gou, Ruohan Wang, Boyuan Zheng, Yanan Xie, Cheng Chang, Yiheng Shu, Huan Sun, and Yu Su · 2024

Cited alongside, same era.

Showui: One vision-language-action model for gui visual agent, 2024

Original

Kevin Qinghong Lin, Linjie Li, Difei Gao, Zhengyuan Yang, Shiwei Wu, Zechen Bai, Weixian Lei, Lijuan Wang, and Mike Zheng Shou · 2024

Cited alongside, same era.

Gpt-4o system card

Original

OpenAI · 2024

Closest in time.

Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

Original

Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin · 2024

Closest in time.

Os-atlas: A foundation action model for generalist gui agents

Original

Zhiyong Wu, Zhenyu Wu, Fangzhi Xu, Yian Wang, Qiushi Sun, Chengyou Jia, Kanzhi Cheng, Zichen Ding, Liheng Chen, Paul Pu Liang, et al · 2024

Closest in time.

Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments, 2024

Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, and Tao Yu · 2024

Closest in time.