Evaluation of Test-Time Adaptation Under Computational Time Constraints

Understand

This paper proposes a novel online evaluation protocol for Test Time Adaptation (TTA) methods, which penalizes slower methods by providing them with fewer samples for adaptation.

• TTA methods leverage unlabeled data at test time to adapt to distribution shifts.
• Although many effective methods have been proposed, their impressive performance usually comes at the cost of significantly increased computation budgets.
• Current evaluation protocols overlook the effect of this extra computation cost, affecting their real-world applicability.