Amortized Neural Networks for Low-Latency Speech Recognition

Understand

We introduce Amortized Neural Networks (AmNets), a compute cost- and latency-aware network architecture particularly well-suited for sequence modeling tasks.

• We apply AmNets to the Recurrent Neural Network Transducer (RNN-T) to reduce compute cost and latency for an automatic speech recognition (ASR) task.
• The AmNets RNN-T architecture enables the network to dynamically switch between encoder branches on a frame-by-frame basis.
• Branches are constructed with variable levels of compute cost and model capacity.