Fetching the paper…

Reading the bibliography…

2021

Leveraging Information Bottleneck for Scientific Document Summarization

Ju, Jiaxin, Liu, Ming, Koh, Huan Yee et al.

Original
PDF
Post on X

Understand

This paper presents an unsupervised extractive approach to summarize scientific long documents based on the Information Bottleneck principle.

Inspired by previous work which uses the Information Bottleneck principle for sentence compression, we extend it to document level summarization with two separate steps.
In the first step, we use signal(s) as queries to retrieve the key content from the source document.
Then, a pre-trained language model conducts further sentence search and edit to return the final extracted summaries.

Leveraging Information Bottleneck for Scientific Document Summarization · Around

Built on

Scibert: Pretrained contextualized embeddings for scientific text
Original
Iz Beltagy, Arman Cohan, and Kyle Lo. 2019 · 1903
Earlier work this paper cites.
Sample efficient text summarization using a single pre-trained transformer
Original
Urvashi Khandelwal, Kevin Clark, Dan Jurafsky, and Lukasz Kaiser. 2019 · 1905
Earlier work this paper cites.
Hibert: Document level pre-training of hierarchical bidirectional transformers for document summarization
Original
Xingxing Zhang, Furu Wei, and Ming Zhou. 2019 · 1905
Earlier work this paper cites.
An entity-driven framework for abstractive summarization
Original
Eva Sharma, Luyang Huang, Zhe Hu, and Lu Wang. 2019 · 1909
Earlier work this paper cites.
Bottlesum: Unsupervised and self-supervised sentence summarization using the information bottleneck principle
Original
Peter West, Ari Holtzman, Jan Buys, and Yejin Choi. 2019 · 1909
Earlier work this paper cites.
Extractive summarization of long documents by combining global and local context
Original
Wen Xiao and Giuseppe Carenini. 2019 · 1909
Earlier work this paper cites.
Summpip: Unsupervised multi-document summarization with sentence graph compression
Jinming Zhao, Ming Liu, Longxiang Gao, Yuan Jin, Lan Du, He Zhao, He Zhang, and Gholamreza Haffari. 2020 · 1952
Earlier work this paper cites.
The information bottleneck method
Naftali Tishby, Fernando C Pereira, and William Bialek. 2000 · 2000
Earlier work this paper cites.
Phase transitions for the information bottleneck in representation learning
Original
Tailin Wu and Ian Fischer. 2020 · 2001
Earlier work this paper cites.
Learning robust representations via multi-view information bottleneck
Original
Marco Federici, Anjan Dutta, Patrick Forré, Nate Kushman, and Zeynep Akata. 2020 · 2002
Earlier work this paper cites.
Abstractive summarization with combination of pre-trained sequence-to-sequence and saliency models
Original
Itsumi Saito, Kyosuke Nishida, Kosuke Nishida, and Junji Tomita. 2020 · 2003
Earlier work this paper cites.
Boosting factual correctness of abstractive summarization with knowledge graph
Original
Chenguang Zhu, William Hinthorn, Ruochen Xu, Qingkai Zeng, Michael Zeng, Xuedong Huang, and Meng Jiang. 2020 · 2003
Earlier work this paper cites.

Similar

Longformer: The long-document transformer
Original
Iz Beltagy, Matthew E Peters, and Arman Cohan. 2020 · 2004
Cited alongside, same era.
Lexrank: Graph-based lexical centrality as salience in text summarization
Günes Erkan and Dragomir R Radev. 2004 · 2004
Cited alongside, same era.
Textrank: Bringing order into text
Rada Mihalcea and Paul Tarau. 2004 · 2004
Cited alongside, same era.
Using latent semantic analysis in text summarization and summary evaluation
Josef Steinberger and Karel Jezek. 2004 · 2004
Cited alongside, same era.
The impact of frequency on summarization
Ani Nenkova and Lucy Vanderwende. 2005 · 2005
Cited alongside, same era.
Beyond sumbasic: Task-focused summarization with sentence simplification and lexical expansion
Lucy Vanderwende, Hisami Suzuki, Chris Brockett, and Ani Nenkova. 2007 · 2007

Then

Neural summarization by extracting sentences and words
Original
Jianpeng Cheng and Mirella Lapata. 2016 · 2016
Later among the works it cites.
Get to the point: Summarization with pointer-generator networks
Original
Abigail See, Peter J Liu, and Christopher D Manning. 2017 · 2017
Later among the works it cites.
A discourse-aware attention model for abstractive summarization of long documents
Original
Arman Cohan, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Seokhwan Kim, Walter Chang, and Nazli Goharian. 2018 · 2018
Later among the works it cites.
Guiding generation for abstractive text summarization based on key information guide network
Chenliang Li, Weiran Xu, Si Li, and Sheng Gao. 2018 · 2018
Later among the works it cites.
Generating wikipedia by summarizing long sequences

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Cited alongside, same era.

Big bird: Transformers for longer sequences

Manzil Zaheer, Guru Guruganesh, Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, et al. 2020 · 2007

Cited alongside, same era.

Enhancing extractive text summarization with topic-aware graph neural networks

Peng Cui, Le Hu, and Yuanchao Liu. 2020 · 2010

Cited alongside, same era.

Automatic keyword extraction from individual documents

Stuart Rose, Dave Engel, Nick Cramer, and Wendy Cowley. 2010 · 2010

Cited alongside, same era.

Sequence to sequence learning with neural networks

Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014 · 2014

Cited alongside, same era.

Variations of the similarity function of textrank for automated summarization

Federico Barrios, Federico López, Luis Argerich, and Rosa Wachenchauzer. 2016 · 2016

Cited alongside, same era.

Summarunner: A recurrent neural network based sequence model for extractive summarization of documents

Ramesh Nallapati, Feifei Zhai, and Bowen Zhou. 2016a

Cited in the paper.

Peter J Liu, Mohammad Saleh, Etienne Pot, Ben Goodrich, Ryan Sepassi, Lukasz Kaiser, and Noam Shazeer. 2018 · 2018

Later among the works it cites.

Meansum: a neural model for unsupervised multi-document abstractive summarization

Eric Chu and Peter Liu. 2019 · 2019

Later among the works it cites.

Intrinsic evaluation of summarization datasets

Rishi Bommasani and Claire Cardie. 2020 · 2020

Later among the works it cites.

Monash-summ@ longsumm 20 scisummpip: An unsupervised scientific paper summarization pipeline

Jiaxin Ju, Ming Liu, Longxiang Gao, and Shirui Pan. 2020 · 2020

Later among the works it cites.

Cord-19: The covid-19 open research dataset

Lucy Lu Wang, Kyle Lo, Yoganand Chandrasekhar, Russell Reas, Jiangjiang Yang, Doug Burdick, Darrin Eide, Kathryn Funk, Yannis Katsis, Rodney Kinney, Yunyao Li, Ziyang Liu, William Merrill, Paul Mooney, Dewey Murdick, Devvret Rishi, Jerry Sheehan, Zhihong Shen, Brandon Stilson, Alex Wade, Kuansan Wang, Nancy Xin Ru Wang, Chris Wilhelm, Boya Xie, Douglas Raymond, Daniel S. Weld, Oren Etzioni, and Sebastian Kohlmeier. 2020 · 2020

Later among the works it cites.

Gsum: A general framework for guided neural abstractive summarization

Zi-Yi Dou, Pengfei Liu, Hiroaki Hayashi, Zhengbao Jiang, and Graham Neubig. 2021 · 2021

Closest in time.

Efficient attentions for long document summarization

Luyang Huang, Shuyang Cao, Nikolaus Parulian, Heng Ji, and Lu Wang. 2021 · 2021

Closest in time.