Can Large Language Models Derive New Knowledge? A Dynamic Benchmark for Biological Knowledge Discovery

Understand

Recent advancements in Large Language Model (LLM) agents have demonstrated remarkable potential in automatic knowledge discovery.

• However, rigorously evaluating an AI's capacity for knowledge discovery remains a critical challenge.
• Existing benchmarks predominantly rely on static datasets, leading to inevitable data contamination where models have likely seen the evaluation knowledge during training.
• Furthermore, the rapid release cycles of modern LLMs render static benchmarks quickly outdated, failing to assess the ability to discover truly new knowledge.