WildFeedback: Aligning LLMs With In-situ User Interactions And Feedback

Understand

As large language models (LLMs) continue to advance, aligning these models with human preferences has emerged as a critical challenge.

• Traditional alignment methods, relying on human or LLM annotated datasets, are limited by their resource-intensive nature, inherent subjectivity, misalignment with real-world user preferences, and the risk of feedback loops that amplify model biases.
• To overcome these limitations, we introduce WildFeedback, a novel framework that leverages in-situ user feedback during conversations with LLMs to create preference datasets automatically.
• Given a corpus of multi-turn user-LLM conversation, WildFeedback identifies and classifies user feedback to LLM responses between conversation turns.