NoisyICL: A Little Noise in Model Parameters Calibrates In-context Learning

Understand

In-Context Learning (ICL) is suffering from unsatisfactory performance and under-calibration due to high prior bias and unfaithful confidence.

• Some previous works fine-tuned language models for better ICL performance with enormous datasets and computing costs.
• In this paper, we propose NoisyICL, simply perturbing the model parameters by random noises to strive for better performance and calibration.
• Our experiments on two models and 12 downstream datasets show that NoisyICL can help ICL produce more accurate predictions.