On Reality and the Limits of Language Data: Aligning LLMs with Human Norms

Understand

Recent advancements in Large Language Models (LLMs) harness linguistic associations in vast natural language data for practical applications.

• However, their ability to understand the physical world using only language data remains a question.
• After reviewing existing protocols, we explore this question using a novel and tightly controlled reasoning test (ART) and compare human norms against versions of GPT-3.
• Our findings highlight the categories of common-sense relations models that could learn directly from data and areas of weakness.