InductionBench: LLMs Fail in the Simplest Complexity Class

Understand

Large language models (LLMs) have shown remarkable improvements in reasoning and many existing benchmarks have been addressed by models such as o1 and o3 either fully or partially.

• However, a majority of these benchmarks emphasize deductive reasoning, including mathematical and coding tasks in which rules such as mathematical axioms or programming syntax are clearly defined, based on which LLMs can plan and apply these rules to arrive at a solution.
• In contrast, inductive reasoning, where one infers the underlying rules from observed data, remains less explored.
• Such inductive processes lie at the heart of scientific discovery, as they enable researchers to extract general principles from empirical observations.