Finding Fast Transformers: One-Shot Neural Architecture Search by Component Composition

Understand

Transformer-based models have achieved stateof-the-art results in many tasks in natural language processing.

• However, such models are usually slow at inference time, making deployment difficult.
• In this paper, we develop an efficient algorithm to search for fast models while maintaining model quality.
• We describe a novel approach to decompose the Transformer architecture into smaller components, and propose a sampling-based one-shot architecture search method to find an optimal model for inference.