The Two Word Test: A Semantic Benchmark for Large Language Models

Understand

Large Language Models (LLMs) have shown remarkable abilities recently, including passing advanced professional exams and demanding benchmark tests.

• This performance has led many to suggest that they are close to achieving humanlike or 'true' understanding of language, and even Artificial General Intelligence (AGI).
• Here, we provide a new open-source benchmark that can assess semantic abilities of LLMs using two-word phrases using a task that can be performed relatively easily by humans without advanced training.
• Combining multiple words into a single concept is a fundamental aspect of human language and intelligence.