Fetching the paper…
Reading the bibliography…
The development of monolingual language models for low and mid-resource languages continues to be hindered by the difficulty in sourcing high-quality training data.
URIEL and lang2vec: Representing languages as typological, geographical, and phylogenetic vectors
Patrick Littell, David R. Mortensen, Ke Lin, Katherine Kairis, Carlisle Turner, and Lori Levin · 2002
Earlier work this paper cites.
A simple, fast, and effective reparameterization of IBM model 2
Chris Dyer, Victor Chahuneau, and Noah A. Smith · 2013
Earlier work this paper cites.
chrF: character n-gram F-score for automatic MT evaluation
Maja Popović · 2015
Earlier work this paper cites.
Neural machine translation of rare words with subword units
Rico Sennrich, Barry Haddow, and Alexandra Birch · 2016
Earlier work this paper cites.
OpenSubtitles2018: Statistical rescoring of sentence alignments in large, noisy parallel corpora
Pierre Lison, Jörg Tiedemann, and Milen Kouylekov · 2018
Earlier work this paper cites.
Investigating backtranslation in neural machine translation
Alberto Poncelas, Dimitar Shterionov, Andy Way, Gideon Maillette de Buy Wenniger, and Peyman Passban · 2018
Earlier work this paper cites.
Know what you don’t know: Unanswerable questions for SQuAD
Pranav Rajpurkar, Robin Jia, and Percy Liang · 2018
Earlier work this paper cites.
Neural fuzzy repair: Integrating fuzzy matches into neural machine translation
Bram Bulte and Arda Tezcan · 2019
Earlier work this paper cites.
Can you compare perplexity across different segmentations?, Apr 2019
Sabrina J. Mielke · 2019
Earlier work this paper cites.
Asynchronous pipelines for processing huge corpora on medium to low resource infrastructures
Pedro Javier Ortiz Suárez, Benoit Sagot, and Laurent Romary · 2019
Earlier work this paper cites.
Exploring the limits of transfer learning with a unified text-to-text transformer
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu · 2019
Earlier work this paper cites.
Translation artifacts in cross-lingual transfer learning
Mikel Artetxe, Gorka Labaka, and Eneko Agirre · 2020
Earlier work this paper cites.
Five new languages now available in google translate, 2020
Google · 2020
Earlier work this paper cites.
The state and fate of linguistic diversity and inclusion in the NLP world
Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury · 2020
Earlier work this paper cites.
A survey of embedding space alignment methods for language and knowledge graphs, 2020
Alexander Kalinowski and Yuan An · 2020
Earlier work this paper cites.
Transformers: State-of-the-art natural language processing
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush · 2020
Earlier work this paper cites.
Adapting monolingual models: Data can be scarce when language similarity is high
Wietse de Vries, Martijn Bartelds, Malvina Nissim, and Martijn Wieling · 2021
Cited alongside, same era.
Towards continual learning for multilingual machine translation via vocabulary substitution
Xavier Garcia, Noah Constant, Ankur Parikh, and Orhan Firat · 2021
Cited alongside, same era.
Overcoming language barriers with microsoft azure translator, Oct 2021
Microsoft · 2021
Cited alongside, same era.
How good is your tokenizer? on the monolingual performance of multilingual language models
Phillip Rust, Jonas Pfeiffer, Ivan Vulić, Sebastian Ruder, and Iryna Gurevych · 2021
Cited alongside, same era.
Towards a cleaner document-oriented multilingual crawled corpus
Julien Abadji, Pedro Ortiz Suarez, Laurent Romary, and Benoît Sagot · 2022
Cited alongside, same era.
Subword-delimited downsampling for better character-level translation
Tokenization impacts multilingual language modeling: Assessing vocabulary allocation and overlap across languages
Tomasz Limisiewicz, Jiří Balhar, and David Mareček · 2023
Later among the works it cites.
Fine-tuning transformers: Vocabulary transfer
Vladislav Mosin, Igor Samenko, Borislav Kozlovskii, Alexey Tikhonov, and Ivan P. Yamshchikov · 2023
Later among the works it cites.
ScandEval: A Benchmark for Scandinavian Natural Language Processing
Dan Saattrup Nielsen · 2023
Later among the works it cites.
Language model tokenizers introduce unfairness between languages
Aleksandar Petrov, Emanuele La Malfa, Philip Torr, and Adel Bibi · 2023
Later among the works it cites.
Tik-to-tok: Translating language models one token at a time: An embedding initialization strategy for efficient language adaptation, 2023
François Remy, Pieter Delobelle, Bettina Berendt, Kris Demuynck, and Thomas Demeester · 2023
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Lukas Edman, Antonio Toral, and Gertjan van Noord · 2022
Cited alongside, same era.
Fast vocabulary transfer for language model compression
Leonidas Gee, Andrea Zugarini, Leonardo Rigutini, and Paolo Torroni · 2022
Cited alongside, same era.
Cross-lingual transfer of monolingual models
Evangelia Gogoulou, Ariel Ekgren, Tim Isbister, and Magnus Sahlgren · 2022
Cited alongside, same era.
LoRA: Low-rank adaptation of large language models
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen · 2022
Cited alongside, same era.
Why don’t people use character-level machine translation?
Jindřich Libovický, Helmut Schmid, and Alexander Fraser · 2022
Cited alongside, same era.
WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models
Benjamin Minixhofer, Fabian Paischer, and Navid Rekabsaz · 2022
Cited alongside, same era.
No language left behind: Scaling human-centered machine translation, 2022
Team NLLB, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shannon Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachko, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, and Jeff Wang · 2022
Cited alongside, same era.
Team MistralAI, Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed · 2023
Later among the works it cites.
Llama 2: Open foundation and fine-tuned chat models
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al · 2023
Later among the works it cites.
Llm in a flash: Efficient large language model inference with limited memory, 2024
Keivan Alizadeh, Iman Mirzadeh, Dmitry Belenko, Karen Khatamifard, Minsik Cho, Carlo C Del Mundo, Mohammad Rastegari, and Mehrdad Farajtabar · 2024
Closest in time.
Tower: An open multilingual large language model for translation-related tasks, 2024
Duarte M. Alves, José Pombal, Nuno M. Guerreiro, Pedro H. Martins, João Alves, Amin Farajian, Ben Peters, Ricardo Rei, Patrick Fernandes, Sweta Agrawal, Pierre Colombo, José G. C. de Souza, and André F. T. Martins · 2024
Closest in time.
Gemma: Open models based on gemini research and technologies
Team Gemma and Google DeepMind · 2024
Closest in time.
Gpt-neo 1.3b dutch model, 2024
Yeb Havinga · 2024
Closest in time.
Mala-500: Massive language adaptation of large language models
Peiqin Lin, Shaoxiong Ji, Jörg Tiedemann, André FT Martins, and Hinrich Schütze · 2024
Closest in time.
Olmo: Accelerating the science of language models, 2024
Team OLMo, Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khyathi Raghavi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Valentina Pyatkin, Abhilasha Ravichander, Dustin Schwenk, Saurabh Shah, Will Smith, Emma Strubell, Nishant Subramani, Mitchell Wortsman, Pradeep Dasigi, Nathan Lambert, Kyle Richardson, Luke Zettlemoyer, Jesse Dodge, Kyle Lo, Luca Soldaini, Noah A. Smith, and Hannaneh Hajishirzi · 2024
Closest in time.
Do llamas work in english? on the latent language of multilingual transformers, 2024
Chris Wendler, Veniamin Veselovsky, Giovanni Monea, and Robert West · 2024
Closest in time.
How do large language models handle multilingualism?, 2024
Yiran Zhao, Wenxuan Zhang, Guizhen Chen, Kenji Kawaguchi, and Lidong Bing · 2024
Closest in time.