Evaluating the Text-to-SQL Capabilities of Large Language Models

Learning to parse database queries using inductive logic programming

John M. Zelle and Raymond J. Mooney. 1996 · 1996

Earlier work this paper cites.

Learning a neural semantic parser from user feedback

Srinivasan Iyer, Ioannis Konstas, Alvin Cheung, Jayant Krishnamurthy, and Luke Zettlemoyer. 2017 · 2017

Earlier work this paper cites.

Improving text-to-SQL evaluation methodology

Catherine Finegan-Dollak, Jonathan K. Kummerfeld, Li Zhang, Karthik Ramanathan, Sesh Sadasivam, Rui Zhang, and Dragomir Radev. 2018 · 2018

Earlier work this paper cites.

Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task

Original

Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir Radev. 2019 · 2019

Earlier work this paper cites.

Language models are few-shot learners

Original

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020 · 2020

Earlier work this paper cites.

Bridging textual and tabular data for cross-domain text-to-sql semantic parsing

Original

Xi Victoria Lin, Richard Socher, and Caiming Xiong. 2020 · 2020

Cited alongside, same era.

Exploring unexplored generalization challenges for cross-database semantic parsing

Alane Suhr, Ming-Wei Chang, Peter Shaw, and Kenton Lee. 2020 · 2020

Cited alongside, same era.

Semantic evaluation for text-to-sql with distilled test suites

Original

Ruiqi Zhong, Tao Yu, and Dan Klein. 2020 · 2020

Cited alongside, same era.

Evaluating large language models trained on code

Original

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. 2021 · 2021

On the Sizes of OpenAI API Models

Leo Gao. 2021 · 2021

Later among the works it cites.

Picard: Parsing incrementally for constrained auto-regressive decoding from language models

Original

Torsten Scholak, Nathan Schucher, and Dzmitry Bahdanau. 2021 · 2021

Later among the works it cites.

Compositional generalization and natural language variation: Can a semantic parsing approach handle both?

Peter Shaw, Ming-Wei Chang, Panupong Pasupat, and Kristina Toutanova. 2021 · 2021

Later among the works it cites.

Research recitation

Original

Albert Ziegler. 2021 · 2021

Later among the works it cites.