PARD: Accelerating LLM Inference with Low-Cost PARallel Draft Model Adaptation

Understand

The autoregressive nature of large language models (LLMs) fundamentally limits inference speed, as each forward pass generates only a single token and is often bottlenecked by memory bandwidth.

• Speculative decoding has emerged as a promising solution, adopting a draft-then-verify strategy to accelerate token generation.
• While the EAGLE series achieves strong acceleration, its requirement of training a separate draft head for each target model introduces substantial adaptation costs.
• In this work, we propose \textbf{PARD (PARallel Draft)}, a novel speculative decoding method featuring \textit{target-independence} and \textit{parallel token prediction}.