Keyword Transformer: A Self-Attention Model for Keyword Spotting

Understand

The Transformer architecture has been successful across many domains, including natural language processing, computer vision and speech recognition.

• In keyword spotting, self-attention has primarily been used on top of convolutional or recurrent encoders.
• We investigate a range of ways to adapt the Transformer architecture to keyword spotting and introduce the Keyword Transformer (KWT), a fully self-attentional architecture that exceeds state-of-the-art performance across multiple tasks without any pre-training or additional data.
• Surprisingly, this simple architecture outperforms more complex models that mix convolutional, recurrent and attentive layers.