Reverse-Engineering Decoding Strategies Given Blackbox Access to a Language Generation System

Language models are few-shot learners

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020 · 1901

Earlier work this paper cites.

Eine wahrscheinlichkeitsaufgabe in der kundenwerbung

George Pólya. 1930 · 1930

Earlier work this paper cites.

Hierarchical neural story generation

Angela Fan, Mike Lewis, and Yann Dauphin. 2018 · 2018

Earlier work this paper cites.

The curious case of neural text degeneration

Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019 · 2019

Earlier work this paper cites.

Language models are unsupervised multitask learners

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019 · 2019

Earlier work this paper cites.

Automatic detection of generated text is easiest when humans are fooled

Daphne Ippolito, Daniel Duckworth, Chris Callison-Burch, and Douglas Eck. 2020 · 2020

Cited alongside, same era.

Reverse engineering configurations of neural text generation models

Yi Tay, Dara Bahri, Che Zheng, Clifford Brunk, Donald Metzler, and Andrew Tomkins. 2020 · 2020

Cited alongside, same era.

Decoding methods in neural language generation: a survey

Sina Zarrieß, Henrik Voigt, and Simeon Schüz. 2021 · 2021

Cited alongside, same era.

Bloom: A 176b-parameter open-access multilingual language model

Original

Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. 2022 · 2022

Cited alongside, same era.

On decoding strategies for neural text generators

Gian Wiher, Clara Meister, and Ryan Cotterell. 2022 · 2022

Later among the works it cites.

Pythia: A suite for analyzing large language models across training and scaling

Original

Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, et al. 2023 · 2023

Closest in time.

On the risks of stealing the decoding algorithms of language models

Original

Ali Naseh, Kalpesh Krishna, Mohit Iyyer, and Amir Houmansadr. 2023 · 2023

Closest in time.

Prompts should not be seen as secrets: Systematically measuring prompt extraction attack success

Original

Yiming Zhang and Daphne Ippolito. 2023 · 2023

Closest in time.