Is It Good Data for Multilingual Instruction Tuning or Just Bad Multilingual Evaluation for Large Language Models?

Translationese in Swedish novels translated from English

Martin Gellerstam. 1986 · 1986

Earlier work this paper cites.

Corpus-based Translation Studies: The Challenges that Lie Ahead

Mona Baker. 1996 · 1996

Earlier work this paper cites.

SQuAD: 100,000+ questions for machine comprehension of text

Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016 · 2016

Earlier work this paper cites.

XNLI: Evaluating cross-lingual sentence representations

Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018 · 2018

Earlier work this paper cites.

XLDA: Cross-lingual data augmentation for natural language inference and question answering

Original

Jasdeep Singh, Bryan McCann, Nitish Shirish Keskar, Caiming Xiong, and Richard Socher. 2019 · 2019

Earlier work this paper cites.

Translation artifacts in cross-lingual transfer learning

Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2020a · 2020

Earlier work this paper cites.

TyDi QA: A benchmark for information-seeking question answering in typologically diverse languages

Jonathan H. Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, and Jennimaria Palomaki. 2020 · 2020

Earlier work this paper cites.

Statistical power and translationese in machine translation evaluation

Yvette Graham, Barry Haddow, and Philipp Koehn. 2020 · 2020

Earlier work this paper cites.

A set of recommendations for assessing human-machine parity in language translation

Samuel Läubli, Sheila Castilho, Graham Neubig, Rico Sennrich, Qinlan Shen, and Antonio Toral. 2020 · 2020

Earlier work this paper cites.

mt5: A massively multilingual pre-trained text-to-text transformer

Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2020 · 2020

Earlier work this paper cites.

Training verifiers to solve math word problems

Original

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021 · 2021

Earlier work this paper cites.

LiRo: Benchmark and leaderboard for Romanian language tasks

Stefan Daniel Dumitrescu, Petru Rebeja, Beata Lorincz, Mihaela Gaman, Andrei Avram, Mihai Ilie, Andrei Pruteanu, Adriana Stan, Lorena Rosia, Cristina Iacobescu, et al. 2021 · 2021

Earlier work this paper cites.

Findings of the 2021 conference on machine translation (WMT21)

Akhbardeh Farhad, Arkhangorodsky Arkady, Biesialska Magdalena, Bojar Ondřej, Chatterjee Rajen, Chaudhary Vishrav, Marta R Costa-jussa, España-Bonet Cristina, Fan Angela, Federmann Christian, et al. 2021 · 2021

Earlier work this paper cites.

Measuring massive multitask language understanding

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021 · 2021

Earlier work this paper cites.

LoRA: Low-rank adaptation of large language models

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022 · 2022

Earlier work this paper cites.

Cross-task generalization via natural language crowdsourcing instructions

Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2022 · 2022

Earlier work this paper cites.

Training language models to follow instructions with human feedback

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022 · 2022

Earlier work this paper cites.

Multitask prompted training enables zero-shot task generalization

Victor Sanh, Albert Webson, Colin Raffel, Stephen H Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, et al. 2022 · 2022

Earlier work this paper cites.

Finetuned language models are zero-shot learners

Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V Le. 2022 · 2022

Earlier work this paper cites.

MEGA: Multilingual evaluation of generative AI

Kabir Ahuja, Harshita Diddee, Rishav Hada, Millicent Ochieng, Krithika Ramesh, Prachi Jain, Akshay Nambi, Tanuja Ganu, Sameer Segal, Mohamed Ahmed, Kalika Bali, and Sunayana Sitaram. 2023 · 2023

Cited alongside, same era.

Spanish pre-trained BERT model and evaluation data

Original

José Luis González Cañete, Gabriel Chaperon, Rodrigo Fuentes, Jou-Hui Ho, Hojin Kang, and Jorge P’erez. 2023 · 2023

Cited alongside, same era.

Free Dolly: Introducing the world’s first truly open instruction-tuned LLM

Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. 2023 · 2023

Cited alongside, same era.

Efficient and effective text encoding for Chinese LLaMA and Alpaca

Original

Yiming Cui, Ziqing Yang, and Xin Yao. 2023 · 2023

Cited alongside, same era.

QLoRA: Efficient finetuning of quantized LLMs

Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023 · 2023

xCoT: Cross-lingual instruction tuning for cross-lingual chain-of-thought reasoning

Original

Linzheng Chai, Jian Yang, Tao Sun, Hongcheng Guo, Jiaheng Liu, Bing Wang, Xiannian Liang, Jiaqi Bai, Tongliang Li, Qiyao Peng, et al. 2024 · 2024

Closest in time.

Monolingual or multilingual instruction tuning: Which makes a better Alpaca

Pinzhen Chen, Shaoxiong Ji, Nikolay Bogoychev, Andrey Kutuzov, Barry Haddow, and Kenneth Heafield. 2024 · 2024

Closest in time.

Zero-shot cross-lingual transfer in instruction tuning of large language models

Nadezhda Chirkova and Vassilina Nikoulina. 2024 · 2024

Closest in time.

The Llama 3 herd of models

Original

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024 · 2024

Closest in time.

BertaQA: How much do language models know about local culture?

Original