2024

Authorship Style Transfer with Policy Optimization

Liu, Shuai, Agarwal, Shantanu, May, Jonathan

Understand

Authorship style transfer aims to rewrite a given text into a specified target while preserving the original meaning in the source.

Existing approaches rely on the availability of a large number of target style exemplars for model training.
However, these overlook cases where a limited number of target style examples are available.
The development of parameter-efficient transfer learning techniques and policy optimization (PO) approaches suggest lightweight PO is a feasible approach to low-resource style transfer.

Built on

Transfertransfo: A transfer learning approach for neural network based conversational agents
Original
Thomas Wolf, Victor Sanh, Julien Chaumond, and Clement Delangue. 2019 · 1901
Earlier work this paper cites.
Ctrl: A conditional transformer language model for controllable generation
Original
Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong, and Richard Socher. 2019 · 1909
Earlier work this paper cites.
Simple statistical gradient-following algorithms for connectionist reinforcement learning
Ronald J. Williams. 1992 · 1992
Earlier work this paper cites.
A monolingual tree-based translation model for sentence simplification
Original
Zhemin Zhu, Delphine Bernhard, and Iryna Gurevych. 2010 · 2010
Earlier work this paper cites.
Paraphrasing for style
Wei Xu, Alan Ritter, Bill Dolan, Ralph Grishman, and Colin Cherry. 2012 · 2012
Earlier work this paper cites.
Proximal policy optimization algorithms
Original
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017 · 2017
Earlier work this paper cites.
Universal sentence encoder
Original
Daniel Matthew Cer, Yinfei Yang, Sheng yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, Yun-Hsuan Sung, Brian Strope, and Ray Kurzweil. 2018 · 2018
Earlier work this paper cites.
Dear sir or madam, may I introduce the GYAFC dataset: Corpus, benchmarks and metrics for formality style transfer
Sudha Rao and Joel Tetreault. 2018 · 2018
Earlier work this paper cites.
Reinforcement learning based text style transfer without parallel training corpus
Hongyu Gong, Suma Bhat, Lingfei Wu, JinJun Xiong, and Wen-mei Hwu. 2019 · 2019
Earlier work this paper cites.
Sentence-BERT: Sentence embeddings using Siamese BERT-networks
Nils Reimers and Iryna Gurevych. 2019 · 2019
Earlier work this paper cites.
Beyond BLEU:training neural machine translation with semantic similarity
John Wieting, Taylor Berg-Kirkpatrick, Kevin Gimpel, and Graham Neubig. 2019 · 2019
Earlier work this paper cites.
The pushshift reddit dataset
Jason Baumgartner, Savvas Zannettou, Brian Keegan, Megan Squire, and Jeremy Blackburn. 2020 · 2020
Earlier work this paper cites.

Similar

Hooks in the headline: Learning to generate headlines with controlled styles
Di Jin, Zhijing Jin, Joey Tianyi Zhou, Lisa Orii, and Peter Szolovits. 2020 · 2020
Cited alongside, same era.
Reformulating unsupervised style transfer as paraphrase generation
Kalpesh Krishna, John Wieting, and Mohit Iyyer. 2020 · 2020
Cited alongside, same era.
Politeness transfer: A tag and generate approach
Aman Madaan, Amrith Setlur, Tanmay Parekh, Barnabas Poczos, Graham Neubig, Yiming Yang, Ruslan Salakhutdinov, Alan W Black, and Shrimai Prabhumoye. 2020 · 2020
Cited alongside, same era.
Unsupervised text style transfer with padded masked language models
Eric Malmi, Aliaksei Severyn, and Sascha Rothe. 2020 · 2020
Cited alongside, same era.
A deep metric learning approach to account linking
Aleem Khan, Elizabeth Fleming, Noah Schofield, Marcus Bishop, and Nicholas Andrews. 2021 · 2021
Cited alongside, same era.
Keep it simple: Unsupervised simplification of multi-paragraph text
Philippe Laban, Tobias Schnabel, Paul Bennett, and Marti A. Hearst. 2021 · 2021
Cited alongside, same era.
On learning text style transfer with direct rewards
Yixin Liu, Graham Neubig, and John Wieting. 2021b · 2021
Cited alongside, same era.
LEWIS: Levenshtein editing for unsupervised text style transfer
Machel Reid and Victor Zhong. 2021 · 2021
Cited alongside, same era.
TextSETTR: Few-shot text style extraction and tunable targeted restyling
Parker Riley, Noah Constant, Mandy Guo, Girish Kumar, David Uthus, and Zarana Parekh. 2021 · 2021
Cited alongside, same era.
Learning universal authorship representations
Rafael A. Rivera-Soto, Olivia Elizabeth Miano, Juanita Ordonez, Barry Y. Chen, Aleem Khan, Marcus Bishop, and Nicholas Andrews. 2021 · 2021
Cited alongside, same era.
A large-scale computational study of content preservation measures for text style transfer and paraphrase generation
Nikolay Babakov, David Dale, Varvara Logacheva, and Alexander Panchenko. 2022 · 2022
Cited alongside, same era.
RLPrompt: Optimizing discrete text prompts with reinforcement learning
Mingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric Xing, and Zhiting Hu. 2022 · 2022
Cited alongside, same era.

Then

LoRA: Low-rank adaptation of large language models
Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022 · 2022
Later among the works it cites.
Improving iterative text revision by learning where to edit from other revision tasks
Zae Myung Kim, Wanyu Du, Vipul Raheja, Dhruv Kumar, and Dongyeop Kang. 2022 · 2022
Later among the works it cites.
Rainier: Reinforced knowledge introspector for commonsense question answering
Jiacheng Liu, Skyler Hallinan, Ximing Lu, Pengfei He, Sean Welleck, Hannaneh Hajishirzi, and Yejin Choi. 2022 · 2022
Later among the works it cites.
Quark: Controllable text generation with reinforced unlearning
Ximing Lu, Sean Welleck, Jack Hessel, Liwei Jiang, Lianhui Qin, Peter West, Prithviraj Ammanabrolu, and Yejin Choi. 2022 · 2022
Later among the works it cites.
A recipe for arbitrary text style transfer with large language models
Emily Reif, Daphne Ippolito, Ann Yuan, Andy Coenen, Chris Callison-Burch, and Jason Wei. 2022 · 2022
Later among the works it cites.
Prompt-and-rerank: A method for zero-shot and few-shot arbitrary textual style transfer with small language models
Mirac Suzgun, Luke Melas-Kyriazi, and Dan Jurafsky. 2022 · 2022
Later among the works it cites.
STEER: Unified style transfer with expert reinforcement
Skyler Hallinan, Faeze Brahman, Ximing Lu, Jaehun Jung, Sean Welleck, and Yejin Choi. 2023 · 2023
Later among the works it cites.
Low-resource authorship style transfer: Can non-famous authors be imitated?
Original
Ajay Patel, Nicholas Andrews, and Chris Callison-Burch. 2023 · 2023
Later among the works it cites.
Direct preference optimization: Your language model is secretly a reward model
Original
Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. 2023 · 2023
Later among the works it cites.
CoEdIT: Text editing by task-specific instruction tuning
Vipul Raheja, Dhruv Kumar, Ryan Koo, and Dongyeop Kang. 2023 · 2023
Later among the works it cites.
Preference ranking optimization for human alignment
Original
Feifan Song, Bowen Yu, Minghao Li, Haiyang Yu, Fei Huang, Yongbin Li, and Houfeng Wang. 2023 · 2023
Later among the works it cites.
Contrastive preference optimization: Pushing the boundaries of llm performance in machine translation
Original
Haoran Xu, Amr Sharaf, Yunmo Chen, Weiting Tan, Lingfeng Shen, Benjamin Van Durme, Kenton Murray, and Young Jin Kim. 2024 · 2024
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…