ChartGPT: Leveraging LLMs to Generate Charts from Abstract Natural Language

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei, “Language models are few-shot learners,” in Proceedings of Advances in Neural Information Processing Systems , vol. 33, 2020, pp. 1877–1901

1901

Earlier work this paper cites.

J. Mackinlay, “Automating the design of graphical presentations of relational information,” Acm Transactions On Graphics , vol. 5, no. 2, pp. 110–141, 1986

1986

Earlier work this paper cites.

J. Mackinlay, “Automating the design of graphical presentations of relational information,” ACM Transactions on Graphics , vol. 5, no. 2, pp. 110–141, 1986

1986

Earlier work this paper cites.

M. Card, Readings in information visualization: using vision to think . Morgan Kaufmann, 1999

1999

Earlier work this paper cites.

R. L. Harris, Information Graphics: A Comprehensive Illustrated Reference . Oxford University Press, USA, 1999

1999

Earlier work this paper cites.

E. H.-h. Chi, “A taxonomy of visualization techniques using the data state reference model,” in Proceedings of IEEE Symposium on Information Visualization , 2000, pp. 69–75

2000

Earlier work this paper cites.

K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in Proceedings of the Annual Meeting of the Association for Computational Linguistics , 2002, pp. 311–318

2002

Earlier work this paper cites.

Tableau Software, “Tableau,” https://www.tableau.com/ , 2003

2003

Earlier work this paper cites.

D. E. Rose and D. Levinson, “Understanding user goals in web search,” in Proceedings of the International Conference on World Wide Web , 2004, pp. 13–19

2004

Earlier work this paper cites.

C.-Y. Lin, “Rouge: A package for automatic evaluation of summaries,” in Text summarization branches out , 2004, pp. 74–81

2004

Earlier work this paper cites.

J. Mackinlay, P. Hanrahan, and C. Stolte, “Show Me: Automatic presentation for visual analysis,” IEEE Transactions on Visualization and Computer Graphics , vol. 13, no. 6, pp. 1137–1144, 2007

2007

Earlier work this paper cites.

Y. Sun, J. Leigh, A. Johnson, and S. Lee, “Articulate: A semi-automated model for translating natural language queries into meaningful visualizations,” in Proceedings of the International Symposium on Smart Graphics , 2010, pp. 184–195

2010

Earlier work this paper cites.

L. Wilkinson, “The grammar of graphics: The ggplot2 package,” in Handbook of computational statistics . Springer, 2012, pp. 375–414

2012

Earlier work this paper cites.

I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” Advances in Neural Information Processing Systems , vol. 27, 2014

2014

Earlier work this paper cites.

Wongsuphasawat, Kanit and Moritz, Dominik and Anand, Anushka and Mackinlay, Jock and Howe, Bill and Heer, Jeffrey, “Voyager: Exploratory analysis via faceted browsing of visualization recommendations,” IEEE Transactions on Visualization and Computer Graphics , vol. 22, no. 1, pp. 649–658, 2015

2015

Earlier work this paper cites.

T. Gao, M. Dontcheva, E. Adar, Z. Liu, and K. G. Karahalios, “DataTone: Managing ambiguity in natural language interfaces for data visualization,” in Proceedings of the Annual Symposium on User Interface Software and Technology , 2015, pp. 489–500

2015

Earlier work this paper cites.

M. Vartak, S. Rahman, S. Madden, A. Parameswaran, and N. Polyzotis, “SeeDB: Efficient data-driven visualization recommendations to support visual analytics,” in Proceedings of the VLDB Endowment International Conference on Very Large Data Bases , vol. 8, no. 13, 2015, p. 2182

2015

Earlier work this paper cites.

A. Satyanarayan, D. Moritz, K. Wongsuphasawat, and J. Heer, “Vega-Lite: A grammar of interactive graphics,” IEEE Transactions on Visualization and Computer Graphics , vol. 23, no. 1, pp. 341–350, 2016

2016

Earlier work this paper cites.

K. Wongsuphasawat, D. Moritz, A. Anand, J. Mackinlay, B. Howe, and J. Heer, “Towards a general-purpose query language for visualization recommendation,” in Proceedings of the Workshop on Human-In-the-Loop Data Analytics , 2016, pp. 1–6

2016

Earlier work this paper cites.

V. Setlur, S. E. Battersby, M. Tory, R. Gossweiler, and A. X. Chang, “Eviza: A natural language interface for visual analysis,” in Proceedings of the Annual Symposium on User Interface Software and Technology , 2016, pp. 365–377

2016

Earlier work this paper cites.

K. Wongsuphasawat, Z. Qu, D. Moritz, R. Chang, F. Ouk, A. Anand, J. Mackinlay, B. Howe, and J. Heer, “Voyager 2: Augmenting visual analysis with partial view specifications,” in Proceedings of the CHI Conference on Human Factors in Computing Systems , 2017, pp. 2648–2659

2017

Earlier work this paper cites.

E. Hoque, V. Setlur, M. Tory, and I. Dykeman, “Applying pragmatics principles for interaction with visual analytics,” IEEE Transactions on Visualization and Computer Graphics , vol. 24, no. 1, pp. 309–318, 2017

2017

Earlier work this paper cites.

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805 , 2018

Original

2018

Earlier work this paper cites.

B. Saket, D. Moritz, H. Lin, V. Dibia, C. Demiralp, and J. Heer, “Beyond heuristics: Learning visualization design,” arXiv preprint arXiv:1807.06641 , 2018

Original

2018

Earlier work this paper cites.

D. Moritz, C. Wang, G. L. Nelson, H. Lin, A. M. Smith, B. Howe, and J. Heer, “Formalizing visualization design knowledge as constraints: Actionable and extensible models in draco,” IEEE Transactions on Visualization and Computer Graphics , vol. 25, no. 1, pp. 438–448, 2018

2018

Earlier work this paper cites.

Y. Luo, X. Qin, N. Tang, and G. Li, “DeepEye: Towards automatic data visualization,” in Proceedings of IEEE International Conference on Data Engineering , 2018, pp. 101–112

2018

Earlier work this paper cites.

L. Battle, P. Duan, Z. Miranda, D. Mukusheva, R. Chang, and M. Stonebraker, “Beagle: Automated extraction and interpretation of visualizations from the web,” in Proceedings of the CHI Conference on Human Factors in Computing Systems , 2018, pp. 1–8

2018

Cited alongside, same era.

V. Setlur, M. Tory, and A. Djalali, “Inferencing underspecified natural language utterances in visual analysis,” in Proceedings of the International Conference on Intelligent User Interfaces , 2019, pp. 40–51

2019

Cited alongside, same era.

S. Welleck, I. Kulikov, S. Roller, E. Dinan, K. Cho, and J. Weston, “Neural text generation with unlikelihood training,” arXiv preprint arXiv:1908.04319 , 2019

Original

2019

Cited alongside, same era.

V. Dibia and Ç. Demiralp, “Data2Vis: Automatic generation of data visualizations using sequence-to-sequence recurrent neural networks,” IEEE Computer Graphics and Applications , vol. 39, no. 5, pp. 33–46, 2019

2019

Cited alongside, same era.

Y. Sun, J. Li, S. Chen, G. Andrienko, N. Andrienko, and K. Zhang, “A learning-based approach for efficient visualization construction,” Visual Informatics , vol. 6, no. 1, pp. 14–25, 2022

2022

Later among the works it cites.

A. Jiang, M. A. Nacenta, and J. Ye, “Visualizations as intermediate representations (vlair): An approach for applying deep learning-based computer vision to non-image-based data,” Visual Informatics , vol. 6, no. 3, pp. 35–50, 2022

2022

Later among the works it cites.

A. Wu, Y. Wang, M. Zhou, X. He, H. Zhang, H. Qu, and D. Zhang, “MultiVision: Designing analytical dashboards with deep learning based recommendation,” IEEE Transactions on Visualization and Computer Graphics , vol. 28, no. 1, pp. 162–172, 2022

2022

Later among the works it cites.

H. Voigt, Ö. Alaçam, M. Meuschke, K. Lawonn, and S. Zarrieß, “The why and the how: A survey on natural language interaction in visualization,” in Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , 2022, pp. 348–374