Early Exiting with Ensemble Internal Classifiers

Understand

As a simple technique to accelerate inference of large-scale pre-trained models, early exiting has gained much attention in the NLP community.

• It allows samples to exit early at internal classifiers without passing through the entire model.
• Most existing work usually trains the internal classifiers independently and employs an exiting strategy to decide whether or not to exit based on the confidence of the current internal classifier.
• However, none of these works takes full advantage of the fact that the internal classifiers are trained to solve the same task therefore can be used to construct an ensemble.