Recursive Speculative Decoding: Accelerating LLM Inference via Sampling Without Replacement

Optimal transport: old and new , volume 338

Villani, C. et al · 2009

Earlier work this paper cites.

Gumbel-max trick and weighted reservoir sampling

Vieira, T · 2014

Earlier work this paper cites.

Attention is all you need

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I · 2017

Earlier work this paper cites.

Findings of the 2018 conference on machine translation (wmt18)

Bojar, O. r., Federmann, C., Fishel, M., Graham, Y., Haddow, B., Huck, M., Koehn, P., and Monz, C · 2018

Earlier work this paper cites.

Don’t give me the details, just the summary! Topic-aware convolutional neural networks for extreme summarization

Original

Narayan, S., Cohen, S. B., and Lapata, M · 2018

Earlier work this paper cites.

The curious case of neural text degeneration

Original

Holtzman, A., Buys, J., Du, L., Forbes, M., and Choi, Y · 2019

Earlier work this paper cites.

Stochastic beams and where to find them: The Gumbel-Top- k k trick for sampling sequences without replacement

Kool, W., Van Hoof, H., and Welling, M · 2019

Earlier work this paper cites.

Fast transformer decoding: One write-head is all you need

Original

Shazeer, N · 2019

Earlier work this paper cites.

Language models are few-shot learners

Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al · 2020

Earlier work this paper cites.

OPT: Open pre-trained transformer language models

Original

Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., et al · 2022

Cited alongside, same era.

GPT-4 technical report

Original

Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al · 2023

Cited alongside, same era.

Medusa: Simple framework for accelerating llm generation with multiple decoding heads

Cai, T., Li, Y., Geng, Z., Peng, H., and Dao, T · 2023

Cited alongside, same era.

Accelerating large language model decoding with speculative sampling

Original

Chen, C., Borgeaud, S., Irving, G., Lespiau, J.-B., Sifre, L., and Jumper, J · 2023

Cited alongside, same era.

Free dolly: Introducing the world’s first truly open instruction-tuned llm, 2023

Fast inference from transformers via speculative decoding

Leviathan, Y., Kalman, M., and Matias, Y · 2023

Later among the works it cites.

SpecInfer: Accelerating generative LLM serving with speculative inference and token tree verification

Original

Miao, X., Oliaro, G., Zhang, Z., Cheng, X., Wang, Z., Wong, R. Y. Y., Chen, Z., Arfeen, D., Abhyankar, R., and Jia, Z · 2023

Later among the works it cites.

SpecTr: Fast speculative decoding via optimal transport

Sun, Z., Suresh, A. T., Ro, J. H., Beirami, A., Jain, H., and Yu, F · 2023

Later among the works it cites.

Llama 2: Open foundation and fine-tuned chat models

Original

Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al · 2023

Later among the works it cites.

A survey on non-autoregressive generation for neural machine translation and beyond

Xiao, Y., Wu, L., Guo, J., Li, J., Zhang, M., Qin, T., and Liu, T.-y · 2023