Fetching the paper…
Reading the bibliography…
Recently, pre-trained transformer-based architectures have proven to be very efficient at language modeling and understanding, given that they are trained on a large enough corpus.
Large batch optimization for deep learning: Training bert in 76 minutes
Yang You, Jing Li, Sashank Reddi, Jonathan Hseu, Sanjiv Kumar, Srinadh Bhojanapalli, Xiaodan Song, James Demmel, Kurt Keutzer, and Cho-Jui Hsieh. 2019 · 1904
Earlier work this paper cites.
Gltr: Statistical detection and visualization of generated text
Sebastian Gehrmann, Hendrik Strobelt, and Alexander M Rush. 2019 · 1906
Earlier work this paper cites.
Roberta: A robustly optimized bert pretraining approach
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019 · 1907
Earlier work this paper cites.
Release strategies and the social impacts of language models
Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert-Voss, Jeff Wu, Alec Radford, Gretchen Krueger, Jong Wook Kim, Sarah Kreps, et al. 2019 · 1908
Earlier work this paper cites.
Ctrl: A conditional transformer language model for controllable generation
Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher. 2019 · 1909
Earlier work this paper cites.
Darts: Dialectal arabic transcription system
Sameer Khurana, Ahmed Ali, and James Glass. 2019 · 1909
Earlier work this paper cites.
Unsupervised cross-lingual representation learning at scale
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019 · 1911
Earlier work this paper cites.
Electra: Pre-training text encoders as discriminators rather than generators
Kevin Clark, Minh-Thang Luong, Quoc V Le, and Christopher D Manning. 2020b · 2003
Earlier work this paper cites.
Language models are few-shot learners
Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020 · 2005
Earlier work this paper cites.
Toward micro-dialect identification in diaglossic and code-switched environments
Muhammad Abdul-Mageed, Chiyu Zhang, AbdelRahim Elmadany, and Lyle Ungar. 2020 · 2010
Earlier work this paper cites.
Parallel data, tools and interfaces in opus
Jörgrg Tiedemann. 2012 · 2012
Earlier work this paper cites.
A complete kaldi recipe for building arabic speech recognition systems
Ahmed Ali, Yifan Zhang, Patrick Cardinal, Najim Dahak, Stephan Vogel, and James Glass. 2014 · 2014
Earlier work this paper cites.
Adam: A method for stochastic optimization
Diederik P Kingma and Jimmy Ba. 2014 · 2014
Earlier work this paper cites.
MADAMIRA: A fast, comprehensive tool for morphological analysis and disambiguation of Arabic
Arfath Pasha, Mohamed Al-Badrashiny, Mona Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014 · 2014
Cited alongside, same era.
1.5 billion words arabic corpus
Ibrahim Abu El-Khair. 2016 · 2016
Cited alongside, same era.
Building a first language model for code-switch arabic-english
Injy Hamed, Mohamed Elmahdy, and Slim Abdennadher. 2017 · 2017
Cited alongside, same era.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Cited alongside, same era.
Universal language model fine-tuning for text classification
Jeremy Howard and Sebastian Ruder. 2018 · 2018
Cited alongside, same era.
Osian: Open source international arabic news corpus-preparation and integration into the clarin-infrastructure
Imad Zeroual, Dirk Goldhahn, Thomas Eckart, and Abdelhak Lakhouaja. 2019 · 2019
Later among the works it cites.
Gpt2-ml: Gpt-2 for multiple languages
Zhibo Zhang. 2019 · 2019
Later among the works it cites.
Arabert: Transformer-based model for arabic language understanding
Wissam Antoun, Fady Baly, and Hazem Hajj. 2020a · 2020
Closest in time.
Improving Arabic text categorization using transformer training diversification
Shammur Absar Chowdhury, Ahmed Abdelali, Kareem Darwish, Jung Soon-Gyo, Joni Salminen, and Bernard J. Jansen. 2020 · 2020
Closest in time.
Making a mini gpt-2 with dialect prompts
Nick Doiron. 2020 · 2020
Closest in time.
Bert transformer model for detecting Arabic GPT2 auto-generated tweets
Fouzi Harrag, Maria Dabbah, Kareem Darwish, and Ahmed Abdelali. 2020 · 2020
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
A call for clarity in reporting BLEU scores
Matt Post. 2018 · 2018
Cited alongside, same era.
Improving language understanding by generative pre-training
Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018 · 2018
Cited alongside, same era.
Adafactor: Adaptive learning rates with sublinear memory cost
Noam Shazeer and Mitchell Stern. 2018 · 2018
Cited alongside, same era.
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Cited alongside, same era.
hulmona: The universal language model in arabic
Obeida ElJundi, Wissam Antoun, Nour El Droubi, Hazem Hajj, Wassim El-Hajj, and Khaled Shaban. 2019 · 2019
Cited alongside, same era.
Neural Arabic question answering
Hussein Mozannar, Elie Maamary, Karl El Hajal, and Hazem Hajj. 2019 · 2019
Cited alongside, same era.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019 · 2019
Cited alongside, same era.
Closest in time.
Automatic detection of machine generated text: A critical survey
Ganesh Jawahar, Muhammad Abdul-Mageed, and Laks Lakshmanan, V.S. 2020 · 2020
Closest in time.
gpt2-small-arabic
Abed Khooli. 2020 · 2020
Closest in time.
Gigabert: Zero-shot transfer learning from english to arabic
Wuwei Lan, Yang Chen, Wei Xu, and Alan Ritter. 2020 · 2020
Closest in time.
Empathy-driven Arabic conversational chatbot
Tarek Naous, Christian Hokayem, and Hazem Hajj. 2020 · 2020
Closest in time.
A monolingual approach to contextualized word embeddings for mid-resource languages
Pedro Javier Ortiz Suárez, Laurent Romary, and Benoît Sagot. 2020 · 2020
Closest in time.
Transformers: State-of-the-art natural language processing
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020 · 2020
Closest in time.
KUISAIL at SemEval-2020 task 12: BERT-CNN for offensive speech identification in social media
Ali Safaya, Moutasem Abdullatif, and Deniz Yuret. 2020 · 2059
Closest in time.