2022

Creative Writing with an AI-Powered Writing Assistant: Perspectives from Professional Writers

Ippolito, Daphne, Yuan, Ann, Coenen, Andy et al.

Understand

Recent developments in natural language generation (NLG) using neural language models have brought us closer than ever to the goal of building AI-powered creative writing tools.

However, most prior work on human-AI collaboration in the creative writing domain has evaluated new systems with amateur writers, typically in contrived user studies of limited scope.
In this work, we commissioned 13 professional, published writers from a diverse set of creative writing backgrounds to craft stories using Wordcraft, a text editor with built-in AI-powered writing assistance tools.
Using interviews and participant journals, we discuss the potential of NLG to have significant impact in the creative writing domain--especially with respect to brainstorming, generation of story details, world-building, and research assistance.

Built on

Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 1901
Earlier work this paper cites.
Automatic novel writing: A status report
Sheldon Klein, John F Aeschlimann, David F Balsiger, Steven L Converse, Mark Foster, Robin Lao, John D Oakley, Joel Smith, et al · 1973
Earlier work this paper cites.
TALE-SPIN, An Interactive Program that Writes Stories.. In International Joint Conference on Artificial Intelligence (IJCAI) , Vol. 77. 91–98
James R Meehan. 1977 · 1977
Earlier work this paper cites.
Story Generation After TALE-SPIN.. In IJCAI , Vol. 81. Citeseer, 16–18
Natlie Dehn. 1981 · 1981
Earlier work this paper cites.
Minstrel: A Computer Model of Creativity and Storytelling
Scott R. Turner. 1993 · 1993
Earlier work this paper cites.
MAKEBELIEVE: Using commonsense knowledge to generate stories. In AAAI/IAAI . 957–958
Hugo Liu and Push Singh. 2002 · 2002
Earlier work this paper cites.
From linear story generation to branching story graphs
Mark O Riedl and Robert Michael Young. 2006 · 2006
Earlier work this paper cites.
Learning to tell tales: A data-driven approach to story generation. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1-Volume 1 . Association for Computational Linguistics, 217–225
Neil McIntyre and Mirella Lapata. 2009 · 2009
Earlier work this paper cites.
I remember
Joe Brainard. 2010 · 2010
Earlier work this paper cites.
Creative help: A story writing assistant. In International Conference on Interactive Digital Storytelling . Springer, 81–92
Melissa Roemmele and Andrew S Gordon. 2015 · 2015
Earlier work this paper cites.
Creative writing with a machine in the loop: Case studies on slogans and stories. In 23rd International Conference on Intelligent User Interfaces . ACM, 329–340
Elizabeth Clark, Anne Spencer Ross, Chenhao Tan, Yangfeng Ji, and Noah A Smith. 2018 · 2018
Earlier work this paper cites.
Hierarchical Neural Story Generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Association for Computational Linguistics, Melbourne, Australia, 889–898
Angela Fan, Mike Lewis, and Yann Dauphin. 2018 · 2018
Earlier work this paper cites.
Transformer-XL: Attentive Language Models beyond a Fixed-Length Context. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics . 2978–2988
Zihang Dai, Zhilin Yang, Yiming Yang, Jaime G Carbonell, Quoc Le, and Ruslan Salakhutdinov. 2019 · 2019
Earlier work this paper cites.

Similar

STORIUM: A Dataset and Evaluation Platform for Machine-in-the-Loop Story Generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) . 6470–6484
Nader Akoury, Shufan Wang, Josh Whiting, Stephen Hood, Nanyun Peng, and Mohit Iyyer. 2020 · 2020
Cited alongside, same era.
Longformer: The long-document transformer
Original
Iz Beltagy, Matthew E Peters, and Arman Cohan. 2020 · 2020
Cited alongside, same era.
How Novelists Use Generative Language Models: An Exploratory User Study. In HAI-GEN+ user2agent@ IUI
Alex Calderwood, Vivian Qiu, Katy Ilonka Gero, and Lydia B Chilton. 2020 · 2020
Cited alongside, same era.
Recent trends in deep learning based open-domain textual question answering systems
Zhen Huang, Shiyi Xu, Minghao Hu, Xinyi Wang, Jinyan Qiu, Yongquan Fu, Yuncai Zhao, Yuxing Peng, and Changjian Wang. 2020 · 2020
Cited alongside, same era.
Collaborative Storytelling with Large-scale Neural Language Models
Eric Nichols, Leo Gao, and Randy Gomez. 2020 · 2020
Cited alongside, same era.
Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21) . 2633–2650
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al · 2021
Cited alongside, same era.
Wordcraft: A human-AI collaborative editor for story writing
Original
Andy Coenen, Luke Davis, Daphne Ippolito, Emily Reif, and Ann Yuan. 2021 · 2021
Cited alongside, same era.
The Power of Scale for Parameter-Efficient Prompt Tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing . 3045–3059
Brian Lester, Rami Al-Rfou, and Noah Constant. 2021 · 2021
Cited alongside, same era.
Prefix-Tuning: Optimizing Continuous Prompts for Generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) . 4582–4597
Xiang Lisa Li and Percy Liang. 2021 · 2021
Cited alongside, same era.
Addressing inquiries about history: An efficient and practical framework for evaluating open-domain chatbot consistency
Original
Zekang Li, Jinchao Zhang, Zhengcong Fei, Yang Feng, and Jie Zhou. 2021 · 2021
Cited alongside, same era.
Inspiration through observation: Demonstrating the influence of automatically generated text on creative writing
Original
Melissa Roemmele. 2021 · 2021
Cited alongside, same era.
Story Centaur: Large Language Model Few Shot Learning as a Creative Writing Tool. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations . 244–256
Ben Swanson, Kory Mathewson, Ben Pietrzak, Sherol Chen, and Monica Dinalescu. 2021 · 2021
Cited alongside, same era.

Then

Quantifying memorization across neural language models
Original
Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. 2022 · 2022
Closest in time.
PaLM: Scaling language modeling with pathways
Original
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al · 2022
Closest in time.
GLAM: Efficient scaling of language models with mixture-of-experts. In International Conference on Machine Learning . PMLR, 5547–5569
Nan Du, Yanping Huang, Andrew M Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, et al · 2022
Closest in time.
General-purpose, long-context autoregressive modeling with Perceiver AR
Original
Curtis Hawthorne, Andrew Jaegle, Cătălina Cangea, Sebastian Borgeaud, Charlie Nash, Mateusz Malinowski, Sander Dieleman, Oriol Vinyals, Matthew Botvinick, Ian Simon, et al · 2022
Closest in time.
Block-recurrent transformers
Original
DeLesley Hutchins, Imanol Schlag, Yuhuai Wu, Ethan Dyer, and Behnam Neyshabur. 2022 · 2022
Closest in time.
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . 8086–8098
Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2022 · 2022
Closest in time.
Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionals
Original
Piotr Mirowski, Kory W Mathewson, Jaylen Pittman, and Richard Evans. 2022 · 2022
Closest in time.
A Recipe for Arbitrary Text Style Transfer with Large Language Models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) . 837–848
Emily Reif, Daphne Ippolito, Ann Yuan, Andy Coenen, Chris Callison-Burch, and Jason Wei. 2022 · 2022
Closest in time.
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Original
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al · 2022
Closest in time.
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Original
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al · 2022
Closest in time.
LaMDA: Language Models for Dialog Applications
Original
Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al · 2022
Closest in time.
Wordcraft: Story Writing With Large Language Models. In 27th International Conference on Intelligent User Interfaces . 841–852
Ann Yuan, Andy Coenen, Emily Reif, and Daphne Ippolito. 2022 · 2022
Closest in time.
OPT: Open Pre-trained Transformer Language Models
Original
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al · 2022
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…