Learning from Natural Language Feedback for Personalized Question Answering

Understand

Personalization is crucial for enhancing both the effectiveness and user satisfaction of language technologies, particularly in information-seeking tasks like question answering.

• Current approaches for personalizing large language models (LLMs) often rely on retrieval-augmented generation (RAG), followed by reinforcement learning with scalar reward signals to teach models how to use retrieved personal context.
• We believe that these scalar rewards sometimes provide weak, non-instructive feedback, limiting learning efficiency and personalization quality.
• We introduce VAC, a novel framework for personalized response generation that replaces scalar rewards with natural language feedback (NLF) that are generated conditioned on the user profiles and the question narratives.