Entity-Based Evaluation of Political Bias in Automatic Summarization

Understand

Growing literature has shown that NLP systems may encode social biases; however, the political bias of summarization models remains relatively unknown.

• In this work, we use an entity replacement method to investigate the portrayal of politicians in automatically generated summaries of news articles.
• We develop an entity-based computational framework to assess the sensitivities of several extractive and abstractive summarizers to the politicians Donald Trump and Joe Biden.
• We find consistent differences in these summaries upon entity replacement, such as reduced emphasis of Trump's presence in the context of the same article and a more individualistic representation of Trump with respect to the collective US government (i.e., administration).