TokenSHAP: Interpreting Large Language Models with Monte Carlo Shapley Value Estimation

Understand

As large language models (LLMs) become increasingly prevalent in critical applications, the need for interpretable AI has grown.

• We introduce TokenSHAP, a novel method for interpreting LLMs by attributing importance to individual tokens or substrings within input prompts.
• This approach adapts Shapley values from cooperative game theory to natural language processing, offering a rigorous framework for understanding how different parts of an input contribute to a model's response.
• TokenSHAP leverages Monte Carlo sampling for computational efficiency, providing interpretable, quantitative measures of token importance.