ByteTransformer: A High-Performance Transformer Boosted for Variable-Length Inputs

Understand

Transformers have become keystone models in natural language processing over the past decade.

• They have achieved great popularity in deep learning applications, but the increasing sizes of the parameter spaces required by transformer models generate a commensurate need to accelerate performance.
• Natural language processing problems are also routinely faced with variable-length sequences, as word counts commonly vary among sentences.
• Existing deep learning frameworks pad variable-length sequences to a maximal length, which adds significant memory and computational overhead.