Measuring and Modifying Factual Knowledge in Large Language Models

Understand

Large Language Models (LLMs) store an extensive amount of factual knowledge obtained from vast collections of text.

• To effectively utilize these models for downstream tasks, it is crucial to have reliable methods for measuring their knowledge.
• However, existing approaches for knowledge measurement have certain limitations, and despite recent efforts, they fail to provide accurate measurements and the necessary insights for modifying the knowledge within LLMs.
• In this work, we employ information theory-based measurements to provide a framework estimating the factual knowledge contained within large language models.