Fetching the paper…

Reading the bibliography…

2025

Speculative Decoding and Beyond: An In-Depth Survey of Techniques

Hu, Yunhai, Liu, Zining, Dong, Zhenyuan et al.

Original
PDF
Post on X

Understand

Sequential dependencies present a fundamental bottleneck in deploying large-scale autoregressive models, particularly for real-time applications.

While traditional optimization approaches like pruning and quantization often compromise model quality, recent advances in generation-refinement frameworks demonstrate that this trade-off can be significantly mitigated.
This survey presents a comprehensive taxonomy of generation-refinement frameworks, analyzing methods across autoregressive sequence tasks.
We categorize methods based on their generation strategies (from simple n-gram prediction to sophisticated draft models) and refinement mechanisms (including single-pass verification and iterative approaches).

Speculative Decoding and Beyond: An In-Depth Survey of Techniques · Around

Built on

D. Hendrycks and K. Gimpel, “Gaussian error linear units (gelus),” arXiv preprint arXiv:1606.08415 , 2016
Original
2016
Earlier work this paper cites.
S. Teerapittayanon, B. McDanel, and H. Kung, “Branchynet: Fast inference via early exiting from deep neural networks,” in 2016 23rd international conference on pattern recognition (ICPR) . IEEE, 2016, pp. 2464–2469
2016
Earlier work this paper cites.
A. Radford, “Improving language understanding by generative pre-training,” 2018
2018
Earlier work this paper cites.
S. Elfwing, E. Uchibe, and K. Doya, “Sigmoid-weighted linear units for neural network function approximation in reinforcement learning,” Neural networks , vol. 107, pp. 3–11, 2018
2018
Earlier work this paper cites.
M. Stern, N. Shazeer, and J. Uszkoreit, “Blockwise parallel decoding for deep autoregressive models,” Advances in Neural Information Processing Systems , vol. 31, 2018
2018
Earlier work this paper cites.
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al. , “Language models are unsupervised multitask learners,” OpenAI blog , vol. 1, no. 8, p. 9, 2019
2019
Earlier work this paper cites.
A. Slivkins et al. , “Introduction to multi-armed bandits,” Foundations and Trends® in Machine Learning , vol. 12, no. 1-2, pp. 1–286, 2019
2019
Earlier work this paper cites.
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al. , “Language models are few-shot learners,” Advances in neural information processing systems , vol. 33, pp. 1877–1901, 2020
2020
Earlier work this paper cites.
J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems , vol. 33, pp. 6840–6851, 2020
2020
Earlier work this paper cites.
H. Liu, Z. Dai, D. So, and Q. V. Le, “Pay attention to mlps,” Advances in neural information processing systems , vol. 34, pp. 9204–9215, 2021
2021
Earlier work this paper cites.
M. Ding, Z. Yang, W. Hong, W. Zheng, C. Zhou, D. Yin, J. Lin, X. Zou, Z. Shao, H. Yang et al. , “Cogview: Mastering text-to-image generation via transformers,” Advances in neural information processing systems , vol. 34, pp. 19 822–19 835, 2021
2021
Earlier work this paper cites.
J. Yu, Y. Xu, J. Y. Koh, T. Luong, G. Baid, Z. Wang, V. Vasudevan, A. Ku, Y. Yang, B. K. Ayan et al. , “Scaling autoregressive models for content-rich text-to-image generation,” arXiv preprint arXiv:2206.10789 , vol. 2, no. 3, p. 5, 2022
Original
2022
Earlier work this paper cites.
Y. Zhuang, Y. Yu, K. Wang, H. Sun, and C. Zhang, “Toolqa: A dataset for llm question answering with external tools,” arXiv preprint arXiv:2306.13304 , 2023
Original
2023
Earlier work this paper cites.
H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar et al. , “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971 , 2023
Original
2023
Earlier work this paper cites.
W. Zhu, H. Liu, Q. Dong, J. Xu, L. Kong, J. Chen, L. Li, and S. Huang, “Multilingual machine translation with large language models: Empirical results and analysis,” arXiv preprint arXiv:2304.04675 , 2023
Original
2023
Earlier work this paper cites.
M. U. Hadi, R. Qureshi, A. Shah, M. Irfan, A. Zafar, M. Shaikh, N. Akhtar, J. Wu, and S. Mirjalili, “A survey on large language models: Applications, challenges, limitations, and practical usage,” TechRxiv , 2023
2023
Earlier work this paper cites.
H. Huang, S. Wu, X. Liang, B. Wang, Y. Shi, P. Wu, M. Yang, and T. Zhao, “Towards making the most of llm for translation quality estimation,” in CCF International Conference on Natural Language Processing and Chinese Computing . Springer, 2023, pp. 375–386
2023
Earlier work this paper cites.
L. Yang, Z. Zhang, Y. Song, S. Hong, R. Xu, Y. Zhao, W. Zhang, B. Cui, and M.-H. Yang, “Diffusion models: A comprehensive survey of methods and applications,” ACM Computing Surveys , vol. 56, no. 4, pp. 1–39, 2023
2023
Earlier work this paper cites.
N. Ding, X. Lv, Q. Wang, Y. Chen, B. Zhou, Z. Liu, and M. Sun, “Sparse low-rank adaptation of pre-trained language models,” arXiv preprint arXiv:2311.11696 , 2023
Original
2023
Earlier work this paper cites.
J. Z. Wu, Y. Ge, X. Wang, S. W. Lei, Y. Gu, Y. Shi, W. Hsu, Y. Shan, X. Qie, and M. Z. Shou, “Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 7623–7633
2023
Earlier work this paper cites.
H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar et al. , “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971 , 2023
Original
2023
Earlier work this paper cites.
H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale et al. , “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288 , 2023
Original
2023
Earlier work this paper cites.
A. Santilli, S. Severino, E. Postolache, V. Maiorca, M. Mancusi, R. Marin, and E. Rodolà, “Accelerating transformer inference for translation via parallel decoding,” arXiv preprint arXiv:2305.10427 , 2023
Original
2023
Earlier work this paper cites.
N. Yang, T. Ge, L. Wang, B. Jiao, D. Jiang, L. Yang, R. Majumder, and F. Wei, “Inference with reference: Lossless acceleration of large language models,” arXiv preprint arXiv:2304.04487 , 2023
Original
2023
Earlier work this paper cites.
Z. He, Z. Zhong, T. Cai, J. D. Lee, and D. He, “Rest: Retrieval-based speculative decoding,” arXiv preprint arXiv:2311.08252 , 2023
Original
2023
Earlier work this paper cites.
H. Xia, T. Ge, P. Wang, S.-Q. Chen, F. Wei, and Z. Sui, “Speculative decoding: Exploiting speculative execution for accelerating seq2seq generation,” in Findings of the Association for Computational Linguistics: EMNLP 2023 , 2023, pp. 3909–3925
2023
Earlier work this paper cites.
X. Liu, L. Hu, P. Bailis, A. Cheung, Z. Deng, I. Stoica, and H. Zhang, “Online speculative decoding,” arXiv preprint arXiv:2310.07177 , 2023
Original
2023
Earlier work this paper cites.
Y. Zhou, K. Lyu, A. S. Rawat, A. K. Menon, A. Rostamizadeh, S. Kumar, J.-F. Kagy, and R. Agarwal, “Distillspec: Improving speculative decoding via knowledge distillation,” arXiv preprint arXiv:2310.08461 , 2023
Original
2023
Earlier work this paper cites.
C. Hooper, S. Kim, H. Mohammadzadeh, H. Genc, K. Keutzer, A. Gholami, and S. Shao, “Speed: Speculative pipelined execution for efficient decoding,” arXiv preprint arXiv:2310.12072 , 2023
Original
2023
Earlier work this paper cites.
S. Bae, J. Ko, H. Song, and S.-Y. Yun, “Fast and robust early-exiting framework for autoregressive language models with synchronized parallel decoding,” arXiv preprint arXiv:2310.05424 , 2023
Original
2023
Earlier work this paper cites.
J. Zhang, J. Wang, H. Li, L. Shou, K. Chen, G. Chen, and S. Mehrotra, “Draft & verify: Lossless large language model acceleration via self-speculative decoding,” arXiv preprint arXiv:2309.08168 , 2023
Original
2023
Earlier work this paper cites.
Y. Leviathan, M. Kalman, and Y. Matias, “Fast inference from transformers via speculative decoding,” in International Conference on Machine Learning . PMLR, 2023, pp. 19 274–19 286
2023
Earlier work this paper cites.
C. Chen, S. Borgeaud, G. Irving, J.-B. Lespiau, L. Sifre, and J. Jumper, “Accelerating large language model decoding with speculative sampling,” arXiv preprint arXiv:2302.01318 , 2023
Original
2023
Earlier work this paper cites.
X. Miao, G. Oliaro, Z. Zhang, X. Cheng, Z. Wang, Z. Zhang, R. Y. Y. Wong, A. Zhu, L. Yang, X. Shi et al. , “Specinfer: Accelerating generative large language model serving with tree-based speculative inference and verification,” arXiv preprint arXiv:2305.09781 , 2023
Original
2023
Earlier work this paper cites.
B. Spector and C. Re, “Accelerating llm inference with staged speculative decoding,” arXiv preprint arXiv:2308.04623 , 2023
Original
2023
Earlier work this paper cites.
Z. Chen, X. Yang, J. Lin, C. Sun, K. C.-C. Chang, and J. Huang, “Cascade speculative drafting for even faster llm inference,” arXiv preprint arXiv:2312.11462 , 2023
Original
2023
Earlier work this paper cites.
S. Yang, G. Lee, J. Cho, D. Papailiopoulos, and K. Lee, “Predictive pipelined decoding: A compute-latency trade-off for exact llm decoding,” arXiv preprint arXiv:2307.05908 , 2023
Original
2023
Earlier work this paper cites.
G. Monea, A. Joulin, and E. Grave, “Pass: Parallel speculative sampling,” arXiv preprint arXiv:2311.13581 , 2023
Original
2023
Earlier work this paper cites.
K. Tian, Y. Jiang, Z. Yuan, B. Peng, and L. Wang, “Visual autoregressive modeling: Scalable image generation via next-scale prediction,” arXiv preprint arXiv:2404.02905 , 2024
Original
2024
Earlier work this paper cites.

Similar

“Open-sora report v1.1,” https://github.com/hpcaitech/Open-Sora/blob/main/docs/report_02.md , 2024
2024
Cited alongside, same era.
R. Svirschevski, A. May, Z. Chen, B. Chen, Z. Jia, and M. Ryabinin, “Specexec: Massively parallel speculative decoding for interactive llm inference on consumer devices,” arXiv preprint arXiv:2406.02532 , 2024
Original
2024
Cited alongside, same era.
D. Xu, W. Yin, H. Zhang, X. Jin, Y. Zhang, S. Wei, M. Xu, and X. Liu, “Edgellm: Fast on-device llm inference with speculative decoding,” IEEE Transactions on Mobile Computing , 2024
2024
Cited alongside, same era.
Z. Wang, R. Zhang, K. Ding, Q. Yang, F. Li, and S. Xiang, “Continuous speculative decoding for autoregressive image generation,” arXiv preprint arXiv:2411.11925 , 2024
Original
2024
Cited alongside, same era.
D. Jang, S. Park, J. Y. Yang, Y. Jung, J. Yun, S. Kundu, S.-Y. Kim, and E. Yang, “Lantern: Accelerating visual autoregressive models with relaxed speculative decoding,” arXiv preprint arXiv:2410.03355 , 2024

Then

S. Zhong, Z. Yang, M. Li, R. Gong, R. Wang, and R. Huang, “Propd: Dynamic token tree pruning and generation for llm parallel decoding,” arXiv preprint arXiv:2402.13485 , 2024
Original
2024
Later among the works it cites.
J. Wang, Y. Su, J. Li, Q. Xia, Z. Ye, X. Duan, Z. Wang, and M. Zhang, “Opt-tree: Speculative decoding with adaptive draft tree structure,” arXiv preprint arXiv:2406.17276 , 2024
Original
2024
Later among the works it cites.
Z. Qin, Z. He, N. Prakriya, J. Cong, and Y. Sun, “Dynamic-width speculative beam decoding for efficient llm inference,” arXiv preprint arXiv:2409.16560 , 2024
Original
2024
Later among the works it cites.
Z. Gong, J. Liu, Z. Wang, P. Wu, J. Wang, X. Cai, D. Zhao, and R. Yan, “Graph-structured speculative decoding,” arXiv preprint arXiv:2407.16207 , 2024
Original
2024
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

2024

Cited alongside, same era.

B. Li, H. Wang, S. Zhang, Y. Guo, and K. Yu, “Fast and high-quality auto-regressive speech synthesis via speculative decoding,” arXiv preprint arXiv:2410.21951 , 2024

2024

Cited alongside, same era.

D. Raj, G. Keren, J. Jia, J. Mahadeokar, and O. Kalinli, “Faster speech-llama inference with multi-token prediction,” arXiv preprint arXiv:2409.08148 , 2024

2024

Cited alongside, same era.

A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Yang, A. Fan et al. , “The llama 3 herd of models,” arXiv preprint arXiv:2407.21783 , 2024

2024

Cited alongside, same era.

Y. Fu, P. Bailis, I. Stoica, and H. Zhang, “Break the sequential dependency of llm inference using lookahead decoding,” arXiv preprint arXiv:2402.02057 , 2024

2024

Cited alongside, same era.

S. Kou, L. Hu, Z. He, Z. Deng, and H. Zhang, “Cllms: Consistency large language models,” arXiv preprint arXiv:2403.00835 , 2024

2024

Cited alongside, same era.

Z. Wang, Z. Wang, L. Le, H. S. Zheng, S. Mishra, V. Perot, Y. Zhang, A. Mattapalli, A. Taly, J. Shang et al. , “Speculative rag: Enhancing retrieval augmented generation through drafting,” arXiv preprint arXiv:2407.08223 , 2024

2024

Cited alongside, same era.

J. Ou, Y. Chen, and W. Tian, “Lossless acceleration of large language model via adaptive n-gram parallel decoding,” arXiv preprint arXiv:2404.08698 , 2024

Original

2024

Cited alongside, same era.

W. Jeon, M. Gagrani, R. Goel, J. Park, M. Lee, and C. Lott, “Recursive speculative decoding: Accelerating llm inference via sampling without replacement,” arXiv preprint arXiv:2402.14160 , 2024

2024

Later among the works it cites.

Y. Cheng, A. Zhang, X. Zhang, C. Wang, and Y. Wang, “Recurrent drafter for fast speculative decoding in large language models,” arXiv preprint arXiv:2403.09919 , 2024

2024

Later among the works it cites.

Y. Xiong, R. Zhang, Y. Li, T. Wu, and L. Zou, “Dyspec: Faster speculative decoding with dynamic token tree structure,” arXiv preprint arXiv:2410.11744 , 2024

2024

Later among the works it cites.

R. Sun, T. Zhou, X. Chen, and L. Sun, “Spechub: Provable acceleration to multi-draft speculative decoding,” arXiv preprint arXiv:2411.05289 , 2024

2024

Later among the works it cites.

A. Khisti, M. R. Ebrahimi, H. Dbouk, A. Behboodi, R. Memisevic, and C. Louizos, “Multi-draft speculative sampling: Canonical architectures and theoretical limits,” arXiv preprint arXiv:2410.18234 , 2024

2024

Later among the works it cites.

H. Narasimhan, W. Jitkrittum, A. S. Rawat, S. Kim, N. Gupta, A. K. Menon, and S. Kumar, “Faster cascades via speculative decoding,” arXiv preprint arXiv:2405.19261 , 2024

2024

Later among the works it cites.

T. Liu, Y. Li, Q. Lv, K. Liu, J. Zhu, and W. Hu, “Parallel speculative decoding with adaptive draft length,” arXiv preprint arXiv:2408.11850 , 2024

2024

Later among the works it cites.

W. Zhao, Y. Huang, X. Han, W. Xu, C. Xiao, X. Zhang, Y. Fang, K. Zhang, Z. Liu, and M. Sun, “Ouroboros: Generating longer drafts phrase by phrase for faster speculative decoding,” in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , Y. Al-Onaizan, M. Bansal, and Y.-N. Chen, Eds. Miami, Florida, USA: Association for Computational Linguistics, Nov. 2024, pp. 13 378–13 393. [Online]. Available: https://aclanthology.org/2024.emnlp-main.742/

2024

Later among the works it cites.

Z. Xiao, H. Zhang, T. Ge, S. Ouyang, V. Ordonez, and D. Yu, “Parallelspec: Parallel drafter for efficient speculative decoding,” arXiv preprint arXiv:2410.05589 , 2024

2024

Later among the works it cites.

H. Yi, F. Lin, H. Li, N. Peiyang, X. Yu, and R. Xiao, “Generation meets verification: Accelerating large language model inference with smart parallel auto-correct decoding,” in Findings of the Association for Computational Linguistics: ACL 2024 , L.-W. Ku, A. Martins, and V. Srikumar, Eds. Bangkok, Thailand: Association for Computational Linguistics, Aug. 2024, pp. 5285–5299. [Online]. Available: https://aclanthology.org/2024.findings-acl.313/

2024

Later among the works it cites.

L. Zhang, Z. Zhang, B. Xu, S. Mei, and D. Li, “Dovetail: A cpu/gpu heterogeneous speculative decoding for llm inference,” arXiv preprint arXiv:2412.18934 , 2024

2024

Later among the works it cites.

C. Li, Z. Zhou, S. Zheng, J. Zhang, Y. Liang, and G. Sun, “Specpim: Accelerating speculative inference on pim-enabled system via architecture-dataflow co-exploration,” in Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3 , 2024, pp. 950–965

2024

Later among the works it cites.

J. Chen, V. Tiwari, R. Sadhukhan, Z. Chen, J. Shi, I. E.-H. Yen, and B. Chen, “Magicdec: Breaking the latency-throughput tradeoff for long context generation with speculative decoding,” arXiv preprint arXiv:2408.11049 , 2024

2024

Later among the works it cites.

H. Qian, S. K. Gonugondla, S. Ha, M. Shang, S. K. Gouda, R. Nallapati, S. Sengupta, X. Ma, and A. Deoras, “Bass: Batched attention-optimized speculative sampling,” arXiv preprint arXiv:2404.15778 , 2024

2024

Later among the works it cites.

Z. Wang, J. Wu, Y. Lai, C. Zhang, and D. Zhou, “Seed: Accelerating reasoning tree construction via scheduled speculative decoding,” arXiv preprint arXiv:2406.18200 , 2024

2024

Later among the works it cites.

B. Butler, S. Yu, A. Mazaheri, and A. Jannesari, “Pipeinfer: Accelerating llm inference using asynchronous pipelined speculation,” in SC24: International Conference for High Performance Computing, Networking, Storage and Analysis . IEEE, 2024, pp. 1–19

2024

Later among the works it cites.

J. Wang, D. Paliotta, A. May, A. M. Rush, and T. Dao, “The mamba in the llama: Distilling and accelerating hybrid models,” arXiv preprint arXiv:2408.15237 , 2024

2024

Later among the works it cites.

W. Xu, R. Han, Z. Wang, L. T. Le, D. Madeka, L. Li, W. Y. Wang, R. Agarwal, C.-Y. Lee, and T. Pfister, “Speculative knowledge distillation: Bridging the teacher-student gap through interleaved sampling,” arXiv preprint arXiv:2410.11325 , 2024

2024

Later among the works it cites.

D. Wagner, S. Lee, I. Baumann, P. Seeberger, K. Riedhammer, and T. Bocklet, “Optimized speculative sampling for gpu hardware accelerators,” arXiv preprint arXiv:2406.11016 , 2024

Original

2024

Later among the works it cites.

Y. Teng, H. Shi, X. Liu, X. Ning, G. Dai, Y. Wang, Z. Li, and X. Liu, “Accelerating auto-regressive text-to-image generation with training-free speculative jacobi decoding,” arXiv preprint arXiv:2410.01699 , 2024

2024

Later among the works it cites.

M. Gagrani, R. Goel, W. Jeon, J. Park, M. Lee, and C. Lott, “On speculative decoding for multimodal large language models,” arXiv preprint arXiv:2404.08856 , 2024

2024

Later among the works it cites.

Y. Xi, H. Wang, B. Chen, J. Lin, M. Zhu, W. Liu, R. Tang, W. Zhang, and Y. Yu, “A decoding acceleration framework for industrial deployable llm-based recommender systems,” arXiv preprint arXiv:2408.05676 , 2024

2024

Later among the works it cites.

X. Lin, C. Yang, W. Wang, Y. Li, C. Du, F. Feng, S.-K. Ng, and T.-S. Chua, “Efficient inference for large language model-based generative recommendation,” arXiv preprint arXiv:2410.05165 , 2024

2024

Later among the works it cites.

A. Liu, B. Feng, B. Xue, B. Wang, B. Wu, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan et al. , “Deepseek-v3 technical report,” arXiv preprint arXiv:2412.19437 , 2024

2024

Later among the works it cites.

O. Brown, Z. Wang, A. Do, N. Mathew, and C. Yu, “Dynamic depth decoding: Faster speculative decoding for llms,” arXiv preprint arXiv:2409.00142 , 2024

2024

Later among the works it cites.

B. McDanel, “Amusd: Asynchronous multi-device speculative decoding for llm acceleration,” arXiv preprint arXiv:2410.17375 , 2024

2024

Later among the works it cites.

H. Sun, Z. Chen, X. Yang, Y. Tian, and B. Chen, “Triforce: Lossless acceleration of long sequence generation with hierarchical speculative decoding,” arXiv preprint arXiv:2404.11912 , 2024

2024

Later among the works it cites.

J. Zhao, W. Lu, S. Wang, L. Kong, and C. Wu, “Qspec: Speculative decoding with complementary quantization schemes,” arXiv preprint arXiv:2410.11305 , 2024

2024

Later among the works it cites.

T. Li, Y. Tian, H. Li, M. Deng, and K. He, “Autoregressive image generation without vector quantization,” arXiv preprint arXiv:2406.11838 , 2024

2024

Later among the works it cites.

J. Liu, B. Park, and X. Shen, “A drop-in solution for on-the-fly adaptation of speculative decoding in large language models,” 2025. [Online]. Available: https://openreview.net/forum?id=xOtOfdbBqK

2025

Closest in time.

G. Bachmann, S. Anagnostidis, A. Pumarola, M. Georgopoulos, A. Sanakoyeu, Y. Du, E. Schönfeld, A. Thabet, and J. K. Kohler, “Judge decoding: Faster speculative sampling requires going beyond model alignment,” in The Thirteenth International Conference on Learning Representations , 2025. [Online]. Available: https://openreview.net/forum?id=mtSSFiqW6y

2025

Closest in time.

G. Liu, A. Ramachandran, T. Gangwani, Y. Fu, and A. Sethy, “Knowledge distillation with training wheels,” 2025. [Online]. Available: https://www.amazon.science/publications/knowledge-distillation-with-training-wheels

2025

Closest in time.

Z. Sun, U. Mendlovic, Y. Leviathan, A. Aharoni, A. Beirami, J. H. Ro, and A. T. Suresh, “Block verification accelerates speculative decoding,” in The Thirteenth International Conference on Learning Representations , 2025. [Online]. Available: https://openreview.net/forum?id=frsg32u0rO

2025

Closest in time.

Z. Hu, T. Zheng, V. Viswanathan, Z. Chen, R. A. Rossi, Y. Wu, D. Manocha, and H. Huang, “Towards optimal multi-draft speculative decoding,” in The Thirteenth International Conference on Learning Representations , 2025. [Online]. Available: https://openreview.net/forum?id=9KxnxWOBA5

2025

Closest in time.