Language Models are Symbolic Learners in Arithmetic

Understand

The prevailing question in LM performing arithmetic is whether these models learn to truly compute or if they simply master superficial pattern matching.

• In this paper, we argues for the latter, presenting evidence that LMs act as greedy symbolic learners, prioritizing the simplest possible shortcuts to fit the stats of dataset to solve arithmetic tasks.
• To investigate this, we introduce subgroup induction, a practical framework adapted from Solomonoff Induction (SI), one of the most powerful universal predictors.
• Our framework analyzes arithmetic problems by breaking them down into subgroups-minimal mappings between a few input digits and a single output digit.