PhyloLM : Inferring the Phylogeny of Large Language Models and Predicting their Performances in Benchmarks

Understand

This paper introduces PhyloLM, a method adapting phylogenetic algorithms to Large Language Models (LLMs) to explore whether and how they relate to each other and to predict their performance characteristics.

• Our method calculates a phylogenetic distance metric based on the similarity of LLMs' output.
• The resulting metric is then used to construct dendrograms, which satisfactorily capture known relationships across a set of 111 open-source and 45 closed models.
• Furthermore, our phylogenetic distance predicts performance in standard benchmarks, thus demonstrating its functional validity and paving the way for a time and cost-effective estimation of LLM capabilities.