Fetching the paper…
Reading the bibliography…
The fixed-size context of Transformer makes GPT models incapable of generating arbitrarily long text.
Sentence-bert: Sentence embeddings using siamese bert-networks
Nils Reimers and Iryna Gurevych · 1908
Earlier work this paper cites.
Finding structure in time
Jeffrey L. Elman · 1990
Earlier work this paper cites.
Long short-term memory
Sepp Hochreiter and Jürgen Schmidhuber · 1997
Earlier work this paper cites.
Story generation with crowdsourced plot graphs
Boyang Li, Stephen Lee-Urban, George Johnston, and Mark Riedl · 2013
Earlier work this paper cites.
Write here, write now! an experimental study of group maintenance in collaborative writing
Jeremy Birnholtz, Stephanie Steinhardt, and Antonella Pavese · 2013
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin · 2017
Earlier work this paper cites.
Hafez: an interactive poetry generation system
Marjan Ghazvininejad, Xing Shi, Jay Priyadarshi, and Kevin Knight · 2017
Earlier work this paper cites.
Improving language understanding by generative pre-training
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al · 2018
Earlier work this paper cites.
Hierarchical neural story generation
Angela Fan, Mike Lewis, and Yann Dauphin · 2018
Earlier work this paper cites.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al · 2019
Earlier work this paper cites.
Transformer-XL: Language modeling with longer-term dependency, 2019
Zihang Dai*, Zhilin Yang*, Yiming Yang, William W. Cohen, Jaime Carbonell, Quoc V. Le, and Ruslan Salakhutdinov · 2019
Earlier work this paper cites.
Generating long sequences with sparse transformers, 2019
Rewon Child, Scott Gray, Alec Radford, and Ilya Sutskever · 2019
Earlier work this paper cites.
R-transformer: Recurrent neural network enhanced transformer, 2019
Zhiwei Wang, Yao Ma, Zitao Liu, and Jiliang Tang · 2019
Cited alongside, same era.
Strategies for structuring story generation
Angela Fan, Mike Lewis, and Yann Dauphin · 2019
Cited alongside, same era.
Neural-based Chinese idiom recommendation for enhancing elegance in essay writing
Yuanchao Liu, Bo Pang, and Bingquan Liu · 2019
Cited alongside, same era.
Plan, write, and revise: an interactive system for open-domain story generation
Seraphina Goldfarb-Tarrant, Haining Feng, and Nanyun Peng · 2019
Cited alongside, same era.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei · 2020
Training language models to follow instructions with human feedback
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Gray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe · 2022
Later among the works it cites.
Recurrent memory transformer
Aydar Bulatov, Yuri Kuratov, and Mikhail Burtsev · 2022
Later among the works it cites.
Coauthor: Designing a human-ai collaborative writing dataset for exploring language model capabilities
Mina Lee, Percy Liang, and Qian Yang · 2022
Later among the works it cites.
Re3: Generating longer stories with recursive reprompting and revision
Kevin Yang, Yuandong Tian, Nanyun Peng, and Dan Klein · 2022
Later among the works it cites.
Opt: Open pre-trained transformer language models
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al · 2022
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Compressive transformers for long-range sequence modelling
Jack W. Rae, Anna Potapenko, Siddhant M. Jayakumar, Chloe Hillier, and Timothy P. Lillicrap · 2020
Cited alongside, same era.
Longformer: The long-document transformer
Iz Beltagy, Matthew E. Peters, and Arman Cohan · 2020
Cited alongside, same era.
Reformer: The efficient transformer
Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya · 2020
Cited alongside, same era.
Big bird: Transformers for longer sequences
Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontañón, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, and Amr Ahmed · 2020
Cited alongside, same era.
Sliding selector network with dynamic memory for extractive summarization of long documents
Peng Cui and Le Hu · 2021
Cited alongside, same era.
Progressive generation of long text with pretrained language models
Bowen Tan, Zichao Yang, Maruan Al-Shedivat, Eric Xing, and Zhiting Hu · 2021
Cited alongside, same era.
Wordcraft: a human-ai collaborative editor for story writing
Andy Coenen, Luke Davis, Daphne Ippolito, Emily Reif, and Ann Yuan · 2021
Cited alongside, same era.
Later among the works it cites.
Train short, test long: Attention with linear biases enables input length extrapolation
Ofir Press, Noah A. Smith, and Mike Lewis · 2022
Later among the works it cites.
Summarize, outline, and elaborate: Long-text generation via hierarchical supervision from extractive summaries
Xiaofei Sun, Zijun Sun, Yuxian Meng, Jiwei Li, and Chun Fan · 2022
Later among the works it cites.
Talebrush: sketching stories with generative pretrained language models
John Joon Young Chung, Wooseok Kim, Kang Min Yoo, Hwaran Lee, Eytan Adar, and Minsuk Chang · 2022
Later among the works it cites.
Zero-shot sonnet generation with discourse-level planning and aesthetics features
Yufei Tian and Nanyun Peng · 2022
Later among the works it cites.
Go back in time: Generating flashbacks in stories with event temporal prompts
Rujun Han, Hong Chen, Yufei Tian, and Nanyun Peng · 2022
Later among the works it cites.
Gpt-4 technical report, 2023
OpenAI · 2023
Closest in time.
Choice over control: How users write with large language models using diegetic and non-diegetic prompting
Hai Dang, Sven Goller, Florian Lehmann, and Daniel Buschek · 2023
Closest in time.