Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both?

Understand

Sequence-to-sequence models excel at handling natural language variation, but have been shown to struggle with out-of-distribution compositional generalization.

• This has motivated new specialized architectures with stronger compositional biases, but most of these approaches have only been evaluated on synthetically-generated datasets, which are not representative of natural language variation.
• In this work we ask: can we develop a semantic parsing approach that handles both natural language variation and compositional generalization? To better assess this capability, we propose new train and test splits of non-synthetic datasets.
• We demonstrate that strong existing approaches do not perform well across a broad set of evaluations.