Evaluating statistical language models as pragmatic reasoners

Understand

The relationship between communicated language and intended meaning is often probabilistic and sensitive to context.

• Numerous strategies attempt to estimate such a mapping, often leveraging recursive Bayesian models of communication.
• In parallel, large language models (LLMs) have been increasingly applied to semantic parsing applications, tasked with inferring logical representations from natural language.
• While existing LLM explorations have been largely restricted to literal language use, in this work, we evaluate the capacity of LLMs to infer the meanings of pragmatic utterances.