Wisdom of Committees: An Overlooked Approach To Faster and More Accurate Models

Understand

Committee-based models (ensembles or cascades) construct models by combining existing pre-trained ones.

• While ensembles and cascades are well-known techniques that were proposed before deep learning, they are not considered a core building block of deep model architectures and are rarely compared to in recent literature on developing efficient models.
• In this work, we go back to basics and conduct a comprehensive analysis of the efficiency of committee-based models.
• We find that even the most simplistic method for building committees from existing, independently pre-trained models can match or exceed the accuracy of state-of-the-art models while being drastically more efficient.