Do Large Language Models Solve ARC Visual Analogies Like People Do?

Understand

The Abstraction Reasoning Corpus (ARC) is a visual analogical reasoning test designed for humans and machines (Chollet, 2019).

• We compared human and large language model (LLM) performance on a new child-friendly set of ARC items.
• Results show that both children and adults outperform most LLMs on these tasks.
• Error analysis revealed a similar "fallback" solution strategy in LLMs and young children, where part of the analogy is simply copied.