EMMA-500: Enhancing Massively Multilingual Adaptation of Large Language Models

Bertscore: Evaluating text generation with bert

Original

Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019 · 1904

Earlier work this paper cites.

The md5 message-digest algorithm

Ronald Rivest. 1992 · 1992

Earlier work this paper cites.

On the resemblance and containment of documents

Andrei Z Broder. 1997 · 1997

Earlier work this paper cites.

Bleu: a method for automatic evaluation of machine translation

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002 · 2002

Earlier work this paper cites.

ROUGE: A package for automatic evaluation of summaries

Chin-Yew Lin. 2004 · 2004

Earlier work this paper cites.

Named entity recognition for south and south East Asian languages: Taking stock

Anil Kumar Singh. 2008 · 2008

Earlier work this paper cites.

W2C – web to corpus – corpora

Martin Majliš. 2011 · 2011

Earlier work this paper cites.

The Kyoto free translation task

Graham Neubig. 2011 · 2011

Earlier work this paper cites.

Mining of massive datasets

Anand Rajaraman and Jeffrey D Ullman. 2011 · 2011

Earlier work this paper cites.

Building large monolingual dictionaries at the Leipzig corpora collection: From 100 to 200 languages

Dirk Goldhahn, Thomas Eckart, and Uwe Quasthoff. 2012 · 2012

Earlier work this paper cites.

The Winograd schema challenge

Hector Levesque, Ernest Davis, and Leora Morgenstern. 2012 · 2012

Earlier work this paper cites.

Parallel data, tools and interfaces in OPUS

Jörg Tiedemann. 2012 · 2012

Earlier work this paper cites.

Evenki life newspaper

Evenki Life. 2014 · 2014

Earlier work this paper cites.

Creating a massively parallel bible corpus

Thomas Mayer and Michael Cysouw. 2014 · 2014

Earlier work this paper cites.

Adam: A method for stochastic optimization

Diederik P. Kingma and Jimmy Ba. 2015 · 2015

Earlier work this paper cites.

chrF: character n-gram F-score for automatic MT evaluation

Maja Popović. 2015 · 2015

Earlier work this paper cites.

A large-scale multilingual disambiguation of glosses

José Camacho-Collados, Claudio Delli Bovi, Alessandro Raganato, and Roberto Navigli. 2016 · 2016

Earlier work this paper cites.

Lsdsem 2017 shared task: The story cloze test

Nasrin Mostafazadeh, Michael Roth, Annie Louis, Nathanael Chambers, and James Allen. 2017 · 2017

Earlier work this paper cites.

Tilde MODEL - multilingual open data for EU languages

Roberts Rozis and Raivis Skadiņš. 2017 · 2017

Earlier work this paper cites.

Attention is all you need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017

Earlier work this paper cites.

Parallel corpora for bi-lingual English-Ethiopian languages statistical machine translation

Solomon Teferra Abate, Michael Melese, Martha Yifiru Tachbelie, Million Meshesha, Solomon Atinafu, Wondwossen Mulugeta, Yaregal Assabie, Hafte Abera, Binyam Ephrem, Tewodros Abebe, Wondimagegnhue Tsegaye, Amanuel Lemma, Tsegaye Andargie, and Seifedin Shifaw. 2018 · 2018

Earlier work this paper cites.

Shami: A corpus of Levantine Arabic dialects

Kathrein Abu Kwaik, Motaz Saad, Stergios Chatzikyriakidis, and Simon Dobnik. 2018 · 2018

Earlier work this paper cites.

Developing new linguistic resources and tools for the Galician language

Rodrigo Agerri, Xavier Gómez Guinovart, German Rigau, and Miguel Anxo Solla Portela. 2018 · 2018

Earlier work this paper cites.

DART: A large dataset of dialectal Arabic tweets

Israa Alsarsour, Esraa Mohamed, Reem Suwaileh, and Tamer Elsayed. 2018 · 2018

Earlier work this paper cites.

Think you have solved question answering? try arc, the ai2 reasoning challenge

Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018 · 2018

Earlier work this paper cites.

Xnli: Evaluating cross-lingual sentence representations

Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel R. Bowman, Holger Schwenk, and Veselin Stoyanov. 2018 · 2018

Earlier work this paper cites.

Arabic dialect identification in the context of bivalency and code-switching

Mahmoud El-Haj, Paul Rayson, and Mariam Aboelezz. 2018 · 2018

Earlier work this paper cites.

The IIT Bombay English-Hindi parallel corpus

Anoop Kunchukuttan, Pratik Mehta, and Pushpak Bhattacharyya. 2018 · 2018

Earlier work this paper cites.

A call for clarity in reporting BLEU scores

Matt Post. 2018 · 2018

Earlier work this paper cites.

Texygen: A benchmarking platform for text generation models

Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. 2018 · 2018

Earlier work this paper cites.

BERT: Pre-training of deep bidirectional transformers for language understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019

Earlier work this paper cites.

Asynchronous pipeline for processing huge corpora on medium to low resource infrastructures

Pedro Javier Ortiz Suárez, Benoît Sagot, and Laurent Romary. 2019 · 2019

Earlier work this paper cites.

Huggingface’s transformers: State-of-the-art natural language processing

Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2019 · 2019

Earlier work this paper cites.

TICO-19: the translation initiative for COvid-19

Antonios Anastasopoulos, Alessandro Cattelan, Zi-Yi Dou, Marcello Federico, Christian Federmann, Dmitriy Genzel, Franscisco Guzmán, Junjie Hu, Macduff Hughes, Philipp Koehn, Rosie Lazar, Will Lewis, Graham Neubig, Mengmeng Niu, Alp Öktem, Eric Paquin, Grace Tang, and Sylwia Tur. 2020 · 2020

Earlier work this paper cites.

ParaCrawl: Web-scale acquisition of parallel corpora

Marta Bañón, Pinzhen Chen, Barry Haddow, Kenneth Heafield, Hieu Hoang, Miquel Esplà-Gomis, Mikel L. Forcada, Amir Kamran, Faheem Kirefu, Philipp Koehn, Sergio Ortiz Rojas, Leopoldo Pla Sempere, Gema Ramírez-Sánchez, Elsa Sarrías, Marek Strelec, Brian Thompson, William Waites, Dion Wiggins, and Jaume Zaragoza. 2020 · 2020

Earlier work this paper cites.

Unsupervised cross-lingual representation learning at scale

Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Édouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020 · 2020

Earlier work this paper cites.

Mapping languages: the corpus of global language use

Jonathan Dunn. 2020 · 2020

Earlier work this paper cites.

Habibi - a multi dialect multi national Arabic song lyrics corpus

Mahmoud El-Haj. 2020 · 2020

Earlier work this paper cites.

IndicNLPSuite: Monolingual corpora, evaluation benchmarks and pre-trained multilingual language models for Indian languages

Divyanshu Kakwani, Anoop Kunchukuttan, Satish Golla, Gokul N.C., Avik Bhattacharyya, Mitesh M. Khapra, and Pratyush Kumar. 2020 · 2020

Earlier work this paper cites.

Towards computational linguistics in Minangkabau language: Studies on sentiment analysis and machine translation

Fajri Koto and Ikhwan Koto. 2020 · 2020

Earlier work this paper cites.

S2ORC: The semantic scholar open research corpus

Kyle Lo, Lucy Lu Wang, Mark Neumann, Rodney Kinney, and Daniel Weld. 2020 · 2020

Earlier work this paper cites.

JParaCrawl: A large scale web-based English-Japanese parallel corpus

Makoto Morishita, Jun Suzuki, and Masaaki Nagata. 2020 · 2020

Earlier work this paper cites.

XCOPA: A multilingual dataset for causal commonsense reasoning

Edoardo Maria Ponti, Goran Glavaš, Olga Majewska, Qianchu Liu, Ivan Vulić, and Anna Korhonen. 2020 · 2020

Earlier work this paper cites.

Exploring the limits of transfer learning with a unified text-to-text transformer

Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020 · 2020

Earlier work this paper cites.

AraBench: Benchmarking dialectal Arabic-English machine translation

Hassan Sajjad, Ahmed Abdelali, Nadir Durrani, and Fahim Dalvi. 2020 · 2020

Cited alongside, same era.

The Tatoeba Translation Challenge – Realistic data sets for low resource and multilingual MT

Jörg Tiedemann. 2020 · 2020

Cited alongside, same era.

CCNet: Extracting high quality monolingual datasets from web crawl data

Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, Vishrav Chaudhary, Francisco Guzmán, Armand Joulin, and Edouard Grave. 2020 · 2020

Cited alongside, same era.

Indonlu: Benchmark and resources for evaluating indonesian natural language understanding

Bryan Wilie, Karissa Vincentio, Genta Indra Winata, Samuel Cahyawijaya, X. Li, Zhi Yuan Lim, S. Soleman, R. Mahendra, Pascale Fung, Syafri Bahar, and A. Purwarianti. 2020 · 2020

Cited alongside, same era.

QADI: Arabic dialect identification in the wild

Ahmed Abdelali, Hamdy Mubarak, Younes Samih, Sabit Hassan, and Kareem Darwish. 2021 · 2021

Cited alongside, same era.

Dataset card for ”project gutenberg”

Manuel Faysse. 2023 · 2023

Later among the works it cites.

A framework for few-shot language model evaluation

Leo Gao, Jonathan Tow, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Kyle McDonell, Niklas Muennighoff, et al. 2023 · 2023

Later among the works it cites.

Glot500: Scaling multilingual corpora and language models to 500 languages

Ayyoob Imani, Peiqin Lin, Amir Hossein Kargaran, Silvia Severini, Masoud Jalili Sabet, Nora Kassner, Chunlan Ma, Helmut Schmid, André F. T. Martins, François Yvon, and Hinrich Schütze. 2023 · 2023

Later among the works it cites.

GlotLID: Language identification for low-resource languages

Amir Hossein Kargaran, Ayyoob Imani, François Yvon, and Hinrich Schütze. 2023 · 2023

Later among the works it cites.

The stack: 3 TB of permissively licensed source code

Denis Kocetkov, Raymond Li, Loubna Ben Allal, Jia Li, Chenghao Mou, Yacine Jernite, Margaret Mitchell, Carlos Muñoz Ferrandis, Sean Hughes, Thomas Wolf, Dzmitry Bahdanau, Leandro von Werra, and Harm de Vries. 2023 · 2023

Later among the works it cites.