Fetching the paper…
Reading the bibliography…
Despite being trained specifically to follow user instructions, today's instructiontuned language models perform poorly when instructed to produce random outputs.
A Maximum Entropy Approach to Natural Language Processing
Adam L. Berger, Stephen A. Della Pietra, and Vincent J. Della Pietra · 1996
Earlier work this paper cites.
Man is to computer programmer as woman is to homemaker? debiasing word embeddings
Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai · 2016
Earlier work this paper cites.
A Simple, Fast Diverse Decoding Algorithm for Neural Generation, December 2016
Jiwei Li, Will Monroe, and Dan Jurafsky · 2016
Earlier work this paper cites.
Classical Structured Prediction Losses for Sequence to Sequence Learning
Sergey Edunov, Myle Ott, Michael Auli, David Grangier, and Marc’Aurelio Ranzato · 2018
Earlier work this paper cites.
Hierarchical Neural Story Generation, May 2018
Angela Fan, Mike Lewis, and Yann Dauphin · 2018
Earlier work this paper cites.
Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models, October 2018
Ashwin K. Vijayakumar, Michael Cogswell, Ramprasath R. Selvaraju, Qing Sun, Stefan Lee, David Crandall, and Dhruv Batra · 2018
Earlier work this paper cites.
Decoupled Weight Decay Regularization
Ilya Loshchilov and Frank Hutter · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever · 2019
Earlier work this paper cites.
Neural Text Generation with Unlikelihood Training, September 2019
Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, and Jason Weston · 2019
Earlier work this paper cites.
The Curious Case of Neural Text Degeneration, February 2020
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi · 2020
Earlier work this paper cites.
Trading Off Diversity and Quality in Natural Language Generation, April 2020
Hugh Zhang, Daniel Duckworth, Daphne Ippolito, and Arvind Neelakantan · 2020
Cited alongside, same era.
Lora: Low-rank adaptation of large language models
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen · 2021
Cited alongside, same era.
Prefix-Tuning: Optimizing Continuous Prompts for Generation
Xiang Lisa Li and Percy Liang · 2021
Cited alongside, same era.
Towards Understanding and Mitigating Social Biases in Language Models
Paul Pu Liang, Chiyu Wu, Louis-Philippe Morency, and Ruslan Salakhutdinov · 2021
Cited alongside, same era.
Generating Datasets with Pretrained Language Models, October 2021
Timo Schick and Hinrich Schütze · 2021
Cited alongside, same era.
Generating Training Data with Language Models: Towards Zero-Shot Language Understanding, October 2022
Yu Meng, Jiaxin Huang, Yu Zhang, and Jiawei Han · 2022
Later among the works it cites.
Mistral 7B, October 2023
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed · 2023
Later among the works it cites.
A Kernel-Based View of Language Model Fine-Tuning, June 2023
Sadhika Malladi, Alexander Wettig, Dingli Yu, Danqi Chen, and Sanjeev Arora · 2023
Later among the works it cites.
Llama 2: Open Foundation and Fine-Tuned Chat Models, July 2023
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom · 2023
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Guy Tevet and Jonathan Berant · 2021
Cited alongside, same era.
Synthbio: A case study in faster curation of text datasets
Ann Yuan, Daphne Ippolito, Vitaly Nikolaev, Chris Callison-Burch, Andy Coenen, and Sebastian Gehrmann · 2021
Cited alongside, same era.
RelationPrompt: Leveraging Prompts to Generate Synthetic Data for Zero-Shot Relation Triplet Extraction
Yew Ken Chia, Lidong Bing, Soujanya Poria, and Luo Si · 2022
Cited alongside, same era.
Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor, December 2022
Or Honovich, Thomas Scialom, Omer Levy, and Timo Schick · 2022
Cited alongside, same era.
WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation, November 2022
Alisa Liu, Swabha Swayamdipta, Noah A. Smith, and Yejin Choi · 2022
Cited alongside, same era.
ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback, October 2022a
Jiacheng Ye, Jiahui Gao, Jiangtao Feng, Zhiyong Wu, Tao Yu, and Lingpeng Kong
Cited in the paper.
ZeroGen: Efficient Zero-shot Learning via Dataset Generation, October 2022b
Jiacheng Ye, Jiahui Gao, Qintong Li, Hang Xu, Jiangtao Feng, Zhiyong Wu, Tao Yu, and Lingpeng Kong
Cited in the paper.
Later among the works it cites.
Self-instruct: Aligning language models with self-generated instructions
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi · 2023
Later among the works it cites.
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena, December 2023
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica · 2023
Later among the works it cites.
Universal and transferable adversarial attacks on aligned language models
Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrikson · 2023
Later among the works it cites.
Gemma: Introducing new state-of-the-art open models
Google · 2024
Closest in time.
TOFU: A Task of Fictitious Unlearning for LLMs, January 2024
Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary C. Lipton, and J. Zico Kolter · 2024
Closest in time.