Confident Adaptive Language Modeling

Understand

Recent advances in Transformer-based large language models (LLMs) have led to significant performance improvements across many tasks.

• These gains come with a drastic increase in the models' size, potentially leading to slow and costly use at inference time.
• In practice, however, the series of generations made by LLMs is composed of varying levels of difficulty.
• While certain predictions truly benefit from the models' full capacity, other continuations are more trivial and can be solved with reduced compute.