2021

nvBench: A Large-Scale Synthesized Dataset for Cross-Domain Natural Language to Visualization Task

Luo, Yuyu, Tang, Jiawei, Li, Guoliang

Understand

NL2VIS - which translates natural language (NL) queries to corresponding visualizations (VIS) - has attracted more and more attention both in commercial visualization vendors and academic researchers.

In the last few years, the advanced deep learning-based models have achieved human-like abilities in many natural language processing (NLP) tasks, which clearly tells us that the deep learning-based technique is a good choice to push the field of NL2VIS.
However, a big balk is the lack of benchmarks with lots of (NL, VIS) pairs.
We present nvBench, the first large-scale NL2VIS benchmark, containing 25,750 (NL, VIS) pairs from 750 tables over 105 domains, synthesized from (NL, SQL) benchmarks to support cross-domain NL2VIS task.

Built on

A multi-modal natural language interface to an information visualization environment
K. Cox, R. E. Grinter, S. L. Hibino, L. J. Jagadeesan, and D. Mantilla · 2001
Earlier work this paper cites.
The effects of word prediction on communication rate for AAC
K. Trnka, D. Yarrington, and et al · 2007
Earlier work this paper cites.
Articulate: A semi-automated model for translating natural language queries into meaningful visualizations
Y. Sun, J. Leigh, A. Johnson, and S. Lee · 2010
Earlier work this paper cites.
The stanford corenlp natural language processing toolkit
C. D. Manning, M. Surdeanu, J. Bauer, J. R. Finkel, S. Bethard, and D. McClosky · 2014
Earlier work this paper cites.
Sequence to sequence learning with neural networks
I. Sutskever, O. Vinyals, and Q. V. Le · 2014
Earlier work this paper cites.
Datatone: Managing ambiguity in natural language interfaces for data visualization
T. Gao, M. Dontcheva, and et al · 2015
Earlier work this paper cites.
Eviza: A natural language interface for visual analysis
V. Setlur, S. E. Battersby, and et al · 2016
Earlier work this paper cites.
Towards visualization recommendation systems
M. Vartak, S. Huang, T. Siddiqui, S. Madden, and A. G. Parameswaran · 2016
Earlier work this paper cites.
Attention is all you need
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin · 2017
Earlier work this paper cites.
Seq2sql: Generating structured queries from natural language using reinforcement learning
Original
V. Zhong and et al · 2017
Earlier work this paper cites.

Similar

Applying pragmatics principles for interaction with visual analytics
E. Hoque, V. Setlur, M. Tory, and I. Dykeman · 2018
Cited alongside, same era.
Deepeye: Towards automatic data visualization
Y. Luo, X. Qin, N. Tang, and G. Li · 2018
Cited alongside, same era.
Deepeye: Creating good data visualizations by keyword search
Y. Luo, X. Qin, N. Tang, G. Li, and X. Wang · 2018
Cited alongside, same era.
Deepeye: Visualizing your data by keyword search
X. Qin, Y. Luo, N. Tang, and G. Li · 2018
Cited alongside, same era.
Vizml: A machine learning approach to visualization recommendation
K. Z. Hu, M. A. Bakker, S. Li, and et al · 2019
Cited alongside, same era.
Towards democratizing relational data visualization
N. Tang, E. Wu, and G. Li · 2019
Cited alongside, same era.
Language models are few-shot learners
T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei · 2020
Cited alongside, same era.
Interactive cleaning for progressive visualization through composite questions
Y. Luo, C. Chai, X. Qin, N. Tang, and G. Li · 2020
Cited alongside, same era.
Visclean: Interactive cleaning for progressive visualization
Y. Luo, C. Chai, X. Qin, N. Tang, and G. Li · 2020
Cited alongside, same era.
Amazon’s QuickSight, https://aws.amazon.com/cn/blogs/aws/amazon-quicksight-q-to-answer-ad-hoc-business-questions/
Cited in the paper.
Ask Data. https://www.tableau.com/products/new-features/ask-data
Cited in the paper.
Microsoft Power BI Q&A. https://docs.microsoft.com/en-us/power-bi/create-reports/power-bi-tutorial-q-and-a
Cited in the paper.

Then

Deeptrack: Monitoring and exploring spatio-temporal data - A case of tracking COVID-19 -
Y. Luo, W. Li, T. Zhao, X. Yu, L. Zhang, G. Li, and N. Tang · 2020
Later among the works it cites.
Steerable self-driving data visualization
Y. Luo, X. Qin, C. Chai, N. Tang, G. Li, and W. Li · 2020
Later among the works it cites.
NL4DV: A toolkit for generating analytic specifications for data visualization from natural language queries
A. Narechania, A. Srinivasan, and J. T. Stasko · 2020
Later among the works it cites.
Making data visualization more efficient and effective: a survey
X. Qin, Y. Luo, N. Tang, and G. Li · 2020
Later among the works it cites.
Interweaving multimodal interaction with flexible unit visualizations for data exploration
A. Srinivasan, B. Lee, and J. T. Stasko · 2020
Later among the works it cites.
Flowsense: A natural language interface for visual data exploration within a dataflow system
B. Yu and C. T. Silva · 2020
Later among the works it cites.
Advisor: Automatic visualization answer for natural-language question on tabular data
C. Liu, Y. Han, R. Jiang, and X. Yuan · 2021
Closest in time.
Synthesizing Natural Language to Visualization (NL2VIS) Benchmarks from NL2SQL Benchmarks
Y. Luo, N. Tang, G. Li, C. Chai, W. Li, and X. Qin · 2021
Closest in time.
Collecting and characterizing natural language utterances for specifying data visualizations
A. Srinivasan, N. Nyapathy, B. Lee, S. M. Drucker, and J. Stasko · 2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…