DesignCoder: Hierarchy-Aware and Self-Correcting UI Code Generation with Large Language Models

Image quality assessment: from error visibility to structural similarity

Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004 · 2004

Earlier work this paper cites.

Faster R-CNN: Towards real-time object detection with region proposal networks

Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2016 · 2016

Earlier work this paper cites.

pix2code: Generating code from a graphical user interface screenshot. In Proceedings of the ACM SIGCHI symposium on engineering interactive computing systems . 1–6

Tony Beltramelli. 2018 · 2018

Earlier work this paper cites.

From ui design image to gui skeleton: a neural machine translator to bootstrap mobile gui implementation. In Proceedings of the 40th International Conference on Software Engineering . 665–676

Chunyang Chen, Ting Su, Guozhu Meng, Zhenchang Xing, and Yang Liu. 2018 · 2018

Earlier work this paper cites.

Sketch2code: Generating a website from a paper mockup

Original

Alex Robinson. 2019 · 2019

Earlier work this paper cites.

Doodle2App: Native app code by freehand UI sketching. In Proceedings of the IEEE/ACM 7th International Conference on Mobile Software Engineering and Systems . 81–84

Soumik Mohian and Christoph Csallner. 2020 · 2020

Earlier work this paper cites.

Screen parsing: Towards reverse engineering of ui models from screenshots. In The 34th Annual ACM Symposium on User Interface Software and Technology . 470–483

Jason Wu, Xiaoyi Zhang, Jeff Nichols, and Jeffrey P Bigham. 2021 · 2021

Earlier work this paper cites.

image2emmet: Automatic code generation from web user interface image

Yong Xu, Lili Bo, Xiaobing Sun, Bin Li, Jing Jiang, and Wei Zhou. 2021 · 2021

Earlier work this paper cites.

Code generation from a graphical user interface via attention-based encoder–decoder model

Wen-Yin Chen, Pavol Podstreleny, Wen-Huang Cheng, Yung-Yao Chen, and Kai-Lung Hua. 2022 · 2022

Earlier work this paper cites.

Psychologically-inspired, unsupervised inference of perceptual groups of GUI widgets from GUI images. In Proceedings of the 30th ACM joint European software engineering conference and symposium on the foundations of software engineering . 332–343

Mulong Xie, Zhenchang Xing, Sidong Feng, Xiwei Xu, Liming Zhu, and Chunyang Chen. 2022 · 2022

Earlier work this paper cites.

UI layers merger: merging UI layers via visual learning and boundary prior

Yunnong Chen, Yankun Zhen, Chuning Shi, Jiazhi Li, Liuqing Chen, Zejian Li, Lingyun Sun, Tingting Zhou, and Yanfang Chang. 2023 · 2023

Earlier work this paper cites.

Designing with Language: Wireframing UI Design Intent with Generative Large Language Models

Original

Sidong Feng, Mingyue Yuan, Jieshan Chen, Zhenchang Xing, and Chunyang Chen. 2023 · 2023

Cited alongside, same era.

Segment anything. In Proceedings of the IEEE/CVF international conference on computer vision . 4015–4026

Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al · 2023

Cited alongside, same era.

Visual instruction tuning

Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023 · 2023

Cited alongside, same era.

Codefun website

2024 · 2024

Cited alongside, same era.

Codia Design to Code

2024 · 2024

Cited alongside, same era.

Figma Software

2024 · 2024

Cited alongside, same era.

Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European Conference on Computer Vision . Springer, 38–55

Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al · 2024

Later among the works it cites.

React Native

Meta. 2024 · 2024

Later among the works it cites.

Design2code: How far are we from automating front-end engineering?

Chenglei Si, Yanzhe Zhang, Zhengyuan Yang, Ruibo Liu, and Diyi Yang. 2024 · 2024

Later among the works it cites.

Automatically generating UI code from screenshot: A divide-and-conquer-based approach

Original

Yuxuan Wan, Chaozheng Wang, Yi Dong, Wenxuan Wang, Shuqing Li, Yintong Huo, and Michael R Lyu. 2024 · 2024

Later among the works it cites.

UICoder: Finetuning Large Language Models to Generate User Interface Code through Automated Feedback. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) . 7504–7518