Can Language Models Falsify? Evaluating Algorithmic Reasoning with Counterexample Creation

Understand

There is growing excitement about the potential of Language Models (LMs) to accelerate scientific discovery.

• Falsifying hypotheses is key to scientific progress, as it allows claims to be iteratively refined over time.
• This process requires significant researcher effort, reasoning, and ingenuity.
• Yet current benchmarks for LMs predominantly assess their ability to generate solutions rather than challenge them.