AdapterHub: A Framework for Adapting Transformers

Understand

The current modus operandi in NLP involves downloading and fine-tuning pre-trained models consisting of millions or billions of parameters.

• Storing and sharing such large trained models is expensive, slow, and time-consuming, which impedes progress towards more general and versatile NLP methods that learn from and for many tasks.
• Adapters -- small learnt bottleneck layers inserted within each layer of a pre-trained model -- ameliorate this issue by avoiding full fine-tuning of the entire model.
• However, sharing and integrating adapter layers is not straightforward.