On Extractive and Abstractive Neural Document Summarization with Transformer Language Models

Understand

We present a method to produce abstractive summaries of long documents that exceed several thousand words via neural abstractive summarization.

• We perform a simple extractive step before generating a summary, which is then used to condition the transformer language model on relevant information before being tasked with generating a summary.
• We show that this extractive step significantly improves summarization results.
• We also show that this approach produces more abstractive summaries compared to prior work that employs a copy mechanism while still achieving higher rouge scores.