2020

CTRLsum: Towards Generic Controllable Text Summarization

He, Junxian, Kryściński, Wojciech, McCann, Bryan et al.

Understand

Current summarization systems yield generic summaries that are disconnected from users' preferences and expectations.

To address this limitation, we present CTRLsum, a novel framework for controllable summarization.
Our approach enables users to control multiple aspects of generated summaries by interacting with the summarization system through textual input in the form of a set of keywords or descriptive prompts.
Using a single unified model, CTRLsum is able to achieve a broad scope of summary manipulation at inference time without requiring additional human annotations or pre-defining a set of control aspects during training.

Built on

Information extraction as a basis for high-precision text classification
Ellen Riloff and Wendy Lehnert · 1994
Earlier work this paper cites.
Interactive document summarization, 1999
Jeremy J Bornstein, Douglass R Cutting, John D Hatton, and Daniel E Rose · 1999
Earlier work this paper cites.
Length-controllable abstractive summarization by guiding with summary prototype
Original
Itsumi Saito, Kyosuke Nishida, Kosuke Nishida, Atsushi Otsuka, Hisako Asano, Junji Tomita, Hiroyuki Shindo, and Yuji Matsumoto · 2001
Earlier work this paper cites.
ineats: interactive multi-document summarization
Anton Leuski, Chin-Yew Lin, and Eduard Hovy · 2003
Earlier work this paper cites.
Abstractive summarization with combination of pre-trained sequence-to-sequence and saliency models
Original
Itsumi Saito, Kyosuke Nishida, Kosuke Nishida, and Junji Tomita · 2003
Earlier work this paper cites.
Rouge: A package for automatic evaluation of summaries
Chin-Yew Lin · 2004
Earlier work this paper cites.
TextRank: Bringing order into text
Rada Mihalcea and Paul Tarau · 2004
Earlier work this paper cites.
Non-expert evaluation of summarization systems is risky
Dan Gillick and Yang Liu · 2010
Earlier work this paper cites.
Teaching machines to read and comprehend
Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom · 2015
Earlier work this paper cites.
Adam: A method for stochastic optimization
Diederik P Kingma and Jimmy Ba · 2015
Earlier work this paper cites.
A neural attention model for abstractive sentence summarization
Alexander M Rush, Sumit Chopra, and Jason Weston · 2015
Earlier work this paper cites.
Neural summarization by extracting sentences and words
Jianpeng Cheng and Mirella Lapata · 2016
Earlier work this paper cites.
Neural interactive translation prediction
Rebecca Knowles and Philipp Koehn · 2016
Earlier work this paper cites.
Sequence to backward and forward sequences: A content-introducing approach to generative short-text conversation
Lili Mou, Yiping Song, Rui Yan, Ge Li, Lu Zhang, and Zhi Jin · 2016
Earlier work this paper cites.
SQuAD: 100,000+ questions for machine comprehension of text
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang · 2016
Earlier work this paper cites.
Chinese poetry generation with planning based neural network
Daisy Zhe Wang, Wei He, Hua Wu, Haiyang Wu, Wei Li, Haifeng Wang, and Enhong Chen · 2016
Earlier work this paper cites.
Models and inference for prefix-constrained machine translation
Joern Wuebker, Spence Green, John DeNero, Saša Hasan, and Minh-Thang Luong · 2016
Earlier work this paper cites.
Convolutional sequence to sequence learning
Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N Dauphin · 2017
Earlier work this paper cites.

Similar

Lexically constrained decoding for sequence generation using grid beam search
Chris Hokamp and Qun Liu · 2017
Cited alongside, same era.
Toward controlled generation of text
Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P Xing · 2017
Cited alongside, same era.
SummaRuNNer: a recurrent neural network based sequence model for extractive summarization of documents
Ramesh Nallapati, Feifei Zhai, and Bowen Zhou · 2017
Cited alongside, same era.
Get to the point: Summarization with pointer-generator networks
Abigail See, Peter J Liu, and Christopher D Manning · 2017
Cited alongside, same era.
Newsqa: A machine comprehension dataset
Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman · 2017
Cited alongside, same era.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin · 2017
Cited alongside, same era.
Machine comprehension using match-lstm and answer pointer
Shuohang Wang and Jing Jiang · 2017
Cited alongside, same era.
A discourse-aware attention model for abstractive summarization of long documents
Arman Cohan, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Seokhwan Kim, Walter Chang, and Nazli Goharian · 2018
Cited alongside, same era.
Bert: Pre-training of deep bidirectional transformers for language understanding
Original
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2018
Cited alongside, same era.
Controllable abstractive summarization
Angela Fan, David Grangier, and Michael Auli · 2018
Cited alongside, same era.
Style transfer in text: Exploration and evaluation
Zhenxin Fu, Xiaoye Tan, Nanyun Peng, Dongyan Zhao, and Rui Yan · 2018
Cited alongside, same era.
Bottom-up abstractive summarization
Sebastian Gehrmann, Yuntian Deng, and Alexander M Rush · 2018
Cited alongside, same era.

Then

A deep reinforced model for abstractive summarization
Romain Paulus, Caiming Xiong, and Richard Socher · 2018
Later among the works it cites.
Fast lexically constrained decoding with dynamic beam allocation for neural machine translation
Matt Post and David Vilar · 2018
Later among the works it cites.
Learning neural templates for text generation
Sam Wiseman, Stuart M Shieber, and Alexander M Rush · 2018
Later among the works it cites.
Hierarchically structured reinforcement learning for topically coherent visual story generation
Qiuyuan Huang, Zhe Gan, Asli Celikyilmaz, Dapeng Wu, Jianfeng Wang, and Xiaodong He · 2019
Later among the works it cites.
Ctrl: A conditional transformer language model for controllable generation
Original
Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher · 2019
Later among the works it cites.
Neural text summarization: A critical evaluation
Wojciech Kryściński, Nitish Shirish Keskar, Bryan McCann, Caiming Xiong, and Richard Socher · 2019
Later among the works it cites.
Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension
Original
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer · 2019
Later among the works it cites.
fairseq: A fast, extensible toolkit for sequence modeling
Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli · 2019
Later among the works it cites.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever · 2019
Later among the works it cites.
BIGPATENT: A large-scale dataset for abstractive and coherent summarization
Eva Sharma, Chen Li, and Lu Wang · 2019
Later among the works it cites.
Target-guided open-domain conversation
Jianheng Tang, Tiancheng Zhao, Chenyan Xiong, Xiaodan Liang, Eric Xing, and Zhiting Hu · 2019
Later among the works it cites.
Huggingface’s transformers: State-of-the-art natural language processing
Original
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush · 2019
Later among the works it cites.
Plan-and-write: Towards better automatic storytelling
Lili Yao, Nanyun Peng, Ralph Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan · 2019
Later among the works it cites.
Pegasus: Pre-training with extracted gap-sentences for abstractive summarization
Original
Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter J Liu · 2019
Later among the works it cites.
Language models are few-shot learners
Original
Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 2020
Closest in time.
Summeval: Re-evaluating summarization evaluation
Original
Alexander R Fabbri, Wojciech Kryściński, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev · 2020
Closest in time.
Spanbert: Improving pre-training by representing and predicting spans
Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S Weld, Luke Zettlemoyer, and Omer Levy · 2020
Closest in time.
BERTScore: Evaluating text generation with bert
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi · 2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…