Fetching the paper…

Reading the bibliography…

2024

ROUTE: Robust Multitask Tuning and Collaboration for Text-to-SQL

Qin, Yang, Chen, Chao, Fu, Zhihang et al.

Original
PDF
Post on X

Understand

Despite the significant advancements in Text-to-SQL (Text2SQL) facilitated by large language models (LLMs), the latest state-of-the-art techniques are still trapped in the in-context learning of closed-source LLMs (e.g., GPT-4), which limits their applicability in open scenarios.

To address this challenge, we propose a novel RObust mUltitask Tuning and collaboration mEthod (ROUTE) to improve the comprehensive capabilities of open-source LLMs for Text2SQL, thereby providing a more practical solution.
Our approach begins with multi-task supervised fine-tuning (SFT) using various synthetic training data related to SQL generation.
Unlike existing SFT-based Text2SQL methods, we introduced several additional SFT tasks, including schema linking, noise correction, and continuation writing.

ROUTE: Robust Multitask Tuning and Collaboration for Text-to-SQL · Around

Built on

Sequence-based structured prediction for semantic parsing
Chunyang Xiao, Marc Dymetman, and Claire Gardent · 2016
Earlier work this paper cites.
Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task
Original
Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, et al · 2018
Earlier work this paper cites.
Representing schema structure with graph neural networks for text-to-sql parsing
Original
Ben Bogin, Matt Gardner, and Jonathan Berant · 2019
Earlier work this paper cites.
Structure-grounded pretraining for text-to-sql
Original
Xiang Deng, Ahmed Hassan Awadallah, Christopher Meek, Oleksandr Polozov, Huan Sun, and Matthew Richardson · 2020
Earlier work this paper cites.
A deep dive into deep learning approaches for text-to-sql systems
George Katsogiannis-Meimarakis and Georgia Koutrika · 2021
Earlier work this paper cites.
Evaluating the text-to-sql capabilities of large language models
Original
Nitarshan Rajkumar, Raymond Li, and Dzmitry Bahdanau · 2022
Earlier work this paper cites.
Gpt-4 technical report
Original
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al · 2023
Earlier work this paper cites.
Dr. spider: A diagnostic evaluation benchmark towards text-to-sql robustness
Original
Shuaichen Chang, Jun Wang, Mingwen Dong, Lin Pan, Henghui Zhu, Alexander Hanbo Li, Wuwei Lan, Sheng Zhang, Jiarong Jiang, Joseph Lilien, et al · 2023
Earlier work this paper cites.
Teaching large language models to self-debug
Original
Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou · 2023
Earlier work this paper cites.
C3: Zero-shot text-to-sql with chatgpt
Original
Xuemei Dong, Chao Zhang, Yuhang Ge, Yuren Mao, Yunjun Gao, Jinshu Lin, Dongfang Lou, et al · 2023
Earlier work this paper cites.
Miga: a unified multi-task generation framework for conversational text-to-sql
Yingwen Fu, Wenjie Ou, Zhou Yu, and Yue Lin · 2023
Earlier work this paper cites.

Similar

Interleaving pre-trained language models and large language models for zero-shot nl2sql generation
Original
Zihui Gu, Ju Fan, Nan Tang, Songyue Zhang, Yuxin Zhang, Zui Chen, Lei Cao, Guoliang Li, Sam Madden, and Xiaoyong Du · 2023
Cited alongside, same era.
Mistral 7b
Original
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al · 2023
Cited alongside, same era.
Efficient memory management for large language model serving with pagedattention
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica · 2023
Cited alongside, same era.
A comprehensive evaluation of chatgpt’s zero-shot text-to-sql capability
Original
Aiwei Liu, Xuming Hu, Lijie Wen, and Philip S Yu · 2023
Cited alongside, same era.

Then

Mcs-sql: Leveraging multiple prompts and multiple-choice selection for text-to-sql generation
Original
Dongjun Lee, Choongwon Park, Jaehyuk Kim, and Heesoo Park · 2024
Closest in time.
The death of schema linking? text-to-sql in the age of well-reasoned language models
Original
Karime Maamari, Fadhil Abubaker, Daniel Jaroslawicz, and Amine Mhedhbi · 2024
Closest in time.
Chase-sql: Multi-path reasoning and preference optimized candidate selection in text-to-sql
Original
Mohammadreza Pourreza, Hailong Li, Ruoxi Sun, Yeounoh Chung, Shayan Talaei, Gaurav Tarlok Kakkar, Yu Gan, Amin Saberi, Fatma Ozcan, and Sercan O Arik · 2024
Closest in time.
Direct preference optimization: Your language model is secretly a reward model
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn · 2024
Closest in time.
Chess: Contextual harnessing for efficient sql synthesis

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Code llama: Open foundation models for code

Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, et al · 2023

Cited alongside, same era.

Exploring chain-of-thought style prompting for text-to-sql

Chang-You Tai, Ziru Chen, Tianshu Zhang, Xiang Deng, and Huan Sun · 2023

Cited alongside, same era.

Llama: Open and efficient foundation language models

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al · 2023

Cited alongside, same era.

Mac-sql: Multi-agent collaboration for text-to-sql

Bing Wang, Changyu Ren, Jian Yang, Xinnian Liang, Jiaqi Bai, Qian-Wen Zhang, Zhao Yan, and Zhoujun Li · 2023

Cited alongside, same era.

Act-sql: In-context learning for text-to-sql with automatically-generated chain-of-thought

Hanchong Zhang, Ruisheng Cao, Lu Chen, Hongshen Xu, and Kai Yu · 2023

Cited alongside, same era.

Deepseek-coder: When the large language model meets programming–the rise of code intelligence

Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al · 2024

Cited alongside, same era.

Towards robustness of text-to-sql models against synonym substitution

Yujian Gan, Xinyun Chen, Qiuping Huang, Matthew Purver, John R Woodward, Jinxia Xie, and Pengsheng Huang

Cited in the paper.

Exploring underexplored limitations of cross-domain text-to-sql generalization

Yujian Gan, Xinyun Chen, and Matthew Purver

Cited in the paper.

Shayan Talaei, Mohammadreza Pourreza, Yu-Chen Chang, Azalia Mirhoseini, and Amin Saberi · 2024

Closest in time.

Qwen2.5: A party of foundation models, September 2024

Qwen Team · 2024

Closest in time.

Bridging language & data: Optimizing text-to-sql generation in large language models, 2024

Niklas Wretblad and Fredrik Gordh Riseby · 2024

Closest in time.

Understanding the effects of noise in text-to-sql: An examination of the bird-bench benchmark

Niklas Wretblad, Fredrik Gordh Riseby, Rahul Biswas, Amin Ahmadi, and Oskar Holmström · 2024

Closest in time.

A survey of table reasoning with large language models

Xuanliang Zhang, Dingzirui Wang, Longxu Dou, Qingfu Zhu, and Wanxiang Che · 2024

Closest in time.

Llamafactory: Unified efficient fine-tuning of 100+ language models

Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma · 2024

Closest in time.