Levels of Analysis for Large Language Models

Understand

Modern artificial intelligence systems, such as large language models, are increasingly powerful but also increasingly hard to understand.

• Recognizing this problem as analogous to the historical difficulties in understanding the human mind, we argue that methods developed in cognitive science can be useful for understanding large language models.
• We propose a framework for applying these methods based on the levels of analysis that David Marr proposed for studying information processing systems.
• By revisiting established cognitive science techniques relevant to each level and illustrating their potential to yield insights into the behavior and internal organization of large language models, we aim to provide a toolkit for making sense of these new kinds of minds.