Accelerating RNN Transducer Inference via One-Step Constrained Beam Search

Understand

We propose a one-step constrained (OSC) beam search to accelerate recurrent neural network (RNN) transducer (RNN-T) inference.

• The original RNN-T beam search has a while-loop leading to speed down of the decoding process.
• The OSC beam search eliminates this while-loop by vectorizing multiple hypotheses.
• This vectorization is nontrivial as the expansion of the hypotheses within the original RNN-T beam search can be different from each other.