Incompressible Knowledge Probes: Estimating Black-Box LLM Parameter Counts via Factual Capacity

Understand

Closed-source frontier labs do not disclose parameter counts.

• Storing F facts requires at least F/(bits per parameter) weights, so factual recall lower-bounds parameter count--an intrinsic, serving-independent signal, though (as we show) a coarse one.
• We introduce Incompressible Knowledge Probes (IKPs), a benchmark of 1,400 factual questions spanning 7 tiers of obscurity, designed to isolate knowledge that cannot be derived by reasoning or compressed by architectural improvements.
• We score with no hallucination penalty (lambda = 0: IKP accuracy is simply the fraction of probed facts answered correctly), which removes both the penalty hyperparameter and the per-tier flooring choice; a full lambda x flooring ablation shows the calibration is robust across scoring choices while individual estimates are not, motivating the no-penalty default.