Fetching the paper…

Reading the bibliography…

2020

BERT Fine-tuning For Arabic Text Summarization

Elmadani, Khalid N., Elgezouli, Mukhtar, Showk, Anas

Original
PDF
Post on X

Understand

Fine-tuning a pretrained BERT model is the state of the art method for extractive/abstractive text summarization, in this paper we showcase how this fine-tuning method can be applied to the Arabic language to both construct the first documented model for abstractive Arabic text summarization and show its performance in Arabic extractive summarization.

Our model works with multilingual BERT (as Arabic language does not have a pretrained BERT of its own).
We show its performance in English corpus first before applying it to Arabic corpora in both extractive and abstractive tasks.

BERT Fine-tuning For Arabic Text Summarization · Around

Built on

Looking for a few good metrics: Rouge and its evaluation
Chin-Yew Lin and FJ Och · 2004
Earlier work this paper cites.
Introduction to arabic natural language processing
Nizar Y Habash · 2010
Earlier work this paper cites.
Teaching machines to read and comprehend
Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom · 2015
Earlier work this paper cites.
Automatic arabic summarization: A survey of methodologies and systems
Lamees Mahmoud Al Qassem, Di Wang, Zaid Al Mahmoud, Hassan Barada, Ahmad Al-Rubaie, and Nawaf I Almoosa · 2017
Earlier work this paper cites.

Similar

Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin · 2017
Cited alongside, same era.
Bert: Pre-training of deep bidirectional transformers for language understanding
Original
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2018
Cited alongside, same era.
Towards a new hybrid approach for abstractive summarization
Younes Jaafar and Karim Bouzoubaa · 2018
Cited alongside, same era.

Then

Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization
Original
Shashi Narayan, Shay B Cohen, and Mirella Lapata · 2018
Later among the works it cites.
Text summarization with pretrained encoders
Original
Yang Liu and Mirella Lapata · 2019
Later among the works it cites.
How multilingual is multilingual bert?
Original
Telmo Pires, Eva Schlinger, and Dan Garrette · 2019
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…