Quadapter: Adapter for GPT-2 Quantization

Understand

Transformer language models such as GPT-2 are difficult to quantize because of outliers in activations leading to a large quantization error.

• To adapt to the error, one must use quantization-aware training, which entails a fine-tuning process based on the dataset and the training pipeline identical to those for the original model.
• Pretrained language models, however, often do not grant access to their datasets and training pipelines, forcing us to rely on arbitrary ones for fine-tuning.
• In that case, it is observed that quantization-aware training overfits the model to the fine-tuning data.