Non-Autoregressive Image Captioning with Counterfactuals-Critical Multi-Agent Learning

Understand

Most image captioning models are autoregressive, i.e.

• they generate each word by conditioning on previously generated words, which leads to heavy latency during inference.
• Recently, non-autoregressive decoding has been proposed in machine translation to speed up the inference time by generating all words in parallel.
• Typically, these models use the word-level cross-entropy loss to optimize each word independently.