ReAGent: A Model-agnostic Feature Attribution Method for Generative Language Models

Understand

Feature attribution methods (FAs), such as gradients and attention, are widely employed approaches to derive the importance of all input features to the model predictions.

• Existing work in natural language processing has mostly focused on developing and testing FAs for encoder-only language models (LMs) in classification tasks.
• However, it is unknown if it is faithful to use these FAs for decoder-only models on text generation, due to the inherent differences between model architectures and task settings respectively.
• Moreover, previous work has demonstrated that there is no `one-wins-all' FA across models and tasks.