FullFront: Benchmarking MLLMs Across the Full Front-End Engineering Workflow

T. Beltramelli, “pix2code: Generating code from a graphical user interface screenshot,” in Proceedings of the ACM SIGCHI symposium on engineering interactive computing systems , 2018, pp. 1–6

2018

Earlier work this paper cites.

X. Chen, Z. Zhao, L. Chen, D. Zhang, J. Ji, A. Luo, Y. Xiong, and K. Yu, “Websrc: a dataset for web-based structural reading comprehension,” arXiv preprint arXiv:2101.09465 , 2021

Original

2021

Earlier work this paper cites.

B. Wang, G. Li, X. Zhou, Z. Chen, T. Grossman, and Y. Li, “Screen2words: Automatic mobile ui summarization with multimodal learning,” in The 34th Annual ACM Symposium on User Interface Software and Technology , 2021, pp. 498–510

2021

Earlier work this paper cites.

A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al. , “Learning transferable visual models from natural language supervision,” in International conference on machine learning . PmLR, 2021, pp. 8748–8763

2021

Earlier work this paper cites.

Y. Chang, M. Narang, H. Suzuki, G. Cao, J. Gao, and Y. Bisk, “Webqa: Multihop and multimodal qa,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 16 495–16 504

2022

Earlier work this paper cites.

X. Deng, Y. Gu, B. Zheng, S. Chen, S. Stevens, B. Wang, H. Sun, and Y. Su, “Mind2web: Towards a generalist agent for the web,” Advances in Neural Information Processing Systems , vol. 36, pp. 28 091–28 114, 2023

2023

Earlier work this paper cites.

H. Guo, W. Zhang, J. Chen, Y. Gu, J. Yang, J. Du, B. Hui, T. Liu, J. Ma, C. Zhou et al. , “Iw-bench: Evaluating large multimodal models for converting image-to-web,” arXiv preprint arXiv:2409.18980 , 2024

Original

2024

Earlier work this paper cites.

M. Wang, S. Sunkara, G. Baechler, J. Lin, Y. Zhu, F. Zubach, L. Shu, and J. Chen, “Webquest: A benchmark for multimodal qa on web page sequences,” arXiv preprint arXiv:2409.13711 , 2024

Original

2024

Earlier work this paper cites.

Z. Wu, C. Han, Z. Ding, Z. Weng, Z. Liu, S. Yao, T. Yu, and L. Kong, “Os-copilot: Towards generalist computer agents with self-improvement,” arXiv preprint arXiv:2402.07456 , 2024

Original

2024

Earlier work this paper cites.

W. Tan, W. Zhang, X. Xu, H. Xia, Z. Ding, B. Li, B. Zhou, J. Yue, J. Jiang, Y. Li et al. , “Cradle: Empowering foundation agents towards general computer control,” arXiv preprint arXiv:2403.03186 , 2024

Original

2024

Earlier work this paper cites.

B. Zheng, B. Gou, J. Kil, H. Sun, and Y. Su, “Gpt-4v (ision) is a generalist web agent, if grounded,” arXiv preprint arXiv:2401.01614 , 2024

Original

2024

Earlier work this paper cites.

O. Yoran, S. J. Amouyal, C. Malaviya, B. Bogin, O. Press, and J. Berant, “Assistantbench: Can web agents solve realistic and time-consuming tasks?” arXiv preprint arXiv:2407.15711 , 2024

Original

2024

Earlier work this paper cites.

K. Cheng, Q. Sun, Y. Chu, F. Xu, Y. Li, J. Zhang, and Z. Wu, “Seeclick: Harnessing gui grounding for advanced visual gui agents,” arXiv preprint arXiv:2401.10935 , 2024

Original

2024

Cited alongside, same era.

D. Chen, Y. Huang, S. Wu, J. Tang, L. Chen, Y. Bai, Z. He, C. Wang, H. Zhou, Y. Li et al. , “Gui-world: A dataset for gui-oriented multimodal llm-based agents,” arXiv e-prints , pp. arXiv–2406, 2024

2024

Cited alongside, same era.

W. Chen, J. Cui, J. Hu, Y. Qin, J. Fang, Y. Zhao, C. Wang, J. Liu, G. Chen, Y. Huo et al. , “Guicourse: From general vision language models to versatile gui agents,” arXiv preprint arXiv:2406.11317 , 2024

Original

2024

Cited alongside, same era.

J. Liu, Y. Song, B. Y. Lin, W. Lam, G. Neubig, Y. Li, and X. Yue, “Visualwebbench: How far have multimodal llms evolved in web page understanding and grounding?” arXiv preprint arXiv:2404.05955 , 2024

Original

2024

Cited alongside, same era.

B. Li, Y. Zhang, D. Guo, R. Zhang, F. Li, H. Zhang, K. Zhang, Y. Li, Z. Liu, and C. Li, “Llava-onevision: Easy visual task transfer,” arXiv preprint arXiv:2408.03326 , 2024

Original

2024

Later among the works it cites.

Y. Gui, Z. Li, Y. Wan, Y. Shi, H. Zhang, B. Chen, Y. Su, D. Chen, S. Wu, X. Zhou et al. , “Webcode2m: A real-world dataset for code generation from webpage designs,” in Proceedings of the ACM on Web Conference 2025 , 2025, pp. 1834–1845

2025

Closest in time.

H. H. Zhao, D. Gao, and M. Z. Shou, “Worldgui: Dynamic testing for comprehensive desktop gui automation,” arXiv preprint arXiv:2502.08047 , 2025

Original

2025

Closest in time.

Z. Wang, H. Xu, J. Wang, X. Zhang, M. Yan, J. Zhang, F. Huang, and H. Ji, “Mobile-agent-e: Self-evolving mobile assistant for complex tasks,” arXiv preprint arXiv:2501.11733 , 2025

Original

2025

Closest in time.