Multi-Source Test-Time Adaptation as Dueling Bandits for Extractive Question Answering

Understand

In this work, we study multi-source test-time model adaptation from user feedback, where K distinct models are established for adaptation.

• To allow efficient adaptation, we cast the problem as a stochastic decision-making process, aiming to determine the best adapted model after adaptation.
• We discuss two frameworks: multi-armed bandit learning and multi-armed dueling bandits.
• Compared to multi-armed bandit learning, the dueling framework allows pairwise collaboration among K models, which is solved by a novel method named Co-UCB proposed in this work.