SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization

Understand

In the summarization domain, a key requirement for summaries is to be factually consistent with the input document.

• Previous work has found that natural language inference (NLI) models do not perform competitively when applied to inconsistency detection.
• In this work, we revisit the use of NLI for inconsistency detection, finding that past work suffered from a mismatch in input granularity between NLI datasets (sentence-level), and inconsistency detection (document level).
• We provide a highly effective and light-weight method called SummaCConv that enables NLI models to be successfully used for this task by segmenting documents into sentence units and aggregating scores between pairs of sentences.