Parameter and Data Efficient Continual Pre-training for Robustness to Dialectal Variance in Arabic

GALE Phase 1 Arabic Broadcast News Parallel Text - Part 1 LDC2007T24. Web Download

Xiaoyi Ma, Dalal Zakhary, and Stephanie Strassel · 2007

Earlier work this paper cites.

GALE Phase 1 Arabic Broadcast News Parallel Text - Part 2 LDC2008T09. Web Download

Xiaoyi Ma, Dalal Zakhary, and Stephanie Strassel · 2008

Earlier work this paper cites.

Morphological analysis and disambiguation for dialectal arabic

Nizar Habash, Ryan Roth, Owen Rambow, Ramy Eskander, and Nadi Tomeh · 2013

Earlier work this paper cites.

Arabic dialect identification

Omar F Zaidan and Chris Callison-Burch · 2014

Earlier work this paper cites.

Decoupled weight decay regularization

Original

Ilya Loshchilov and Frank Hutter · 2017

Earlier work this paper cites.

BERT: Pre-training of deep bidirectional transformers for language understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2019

Cited alongside, same era.

Cross-lingual language model pretraining

Alexis CONNEAU and Guillaume Lample · 2019

Cited alongside, same era.

How multilingual is multilingual bert?

Original

Telmo Pires, Eva Schlinger, and Dan Garrette · 2019

Cited alongside, same era.

What does bert learn about the structure of language?

Ganesh Jawahar, Benoît Sagot, and Djamé Seddah · 2019

Cited alongside, same era.

Arabert: Transformer-based model for arabic language understanding

Original

Wissam Antoun, Fady Baly, and Hazem Hajj · 2020

Later among the works it cites.

Alue: Arabic language understanding evaluation

Haitham Seelawi, Ibraheem Tuffaha, Mahmoud Gzawi, Wael Farhan, Bashar Talafha, Riham Badawi, Zyad Sober, Oday Al-Dweik, Abed Alhakim Freihat, and Hussein Al-Natsheh · 2021

Later among the works it cites.

BOLT Egyptian Arabic SMS/Chat Parallel Training Data LDC2021T15. Web Download

Jennifer Tracey et al · 2021

Later among the works it cites.

Match the script, adapt if multilingual: Analyzing the effect of multilingual pretraining on cross-lingual transferability

Original

Yoshinari Fujinuma, Jordan Boyd-Graber, and Katharina Kann · 2022

Closest in time.