Diff-eRank: A Novel Rank-Based Metric for Evaluating Large Language Models

Understand

Large Language Models (LLMs) have transformed natural language processing and extended their powerful capabilities to multi-modal domains.

• As LLMs continue to advance, it is crucial to develop diverse and appropriate metrics for their evaluation.
• In this paper, we introduce a novel rank-based metric, Diff-eRank, grounded in information theory and geometry principles.
• Diff-eRank assesses LLMs by analyzing their hidden representations, providing a quantitative measure of how efficiently they eliminate redundant information during training.