Continuous Speculative Decoding for Autoregressive Image Generation

Understand

Continuous visual autoregressive (AR) models have demonstrated promising performance in image generation, but their inherently sequential nature results in slow inference speed.

• Speculative decoding, a successful acceleration technique for large language models (LLMs), has effectively accelerated discrete visual AR models.
• However, the absence of an analogous theory for continuous distributions precludes its use in accelerating continuous AR models.
• To fill this gap, this work presents continuous speculative decoding, and addresses challenges from: 1) low acceptance rate, caused by inconsistent output distribution modeled by target and draft models, and 2) modified distribution without analytic expression, caused by a complex integral.