BEAVER: An Enterprise Benchmark for Text-to-SQL

Understand

Existing text-to-SQL benchmarks have largely been constructed from public databases with well-structured schemas and simplistic question-SQL pairs.

• While large language models (LLMs) excel on these settings, their efficacy in complex private enterprise environments, characterized by intricate schemas, domain knowledge, and analytical user queries involving sophisticated structures and functions, remains unproven.
• To bridge this gap, we introduce BEAVER, the first text-to-SQL benchmark derived from private data warehouses.
• It comprises 9128 question-SQL pairs sourced from real-world query logs and 812 tables across 19 diverse domains.