Fetching the paper…
Reading the bibliography…
Adapting English-based large language models (LLMs) to other languages has become increasingly popular due to the efficiency and potential of cross-lingual transfer.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020 · 1901
Earlier work this paper cites.
Development of a question answering system focused on an encyclopedia [百科事典を対象とした質問応答システムの開発] (in Japanese)
Satoshi Sekine. 2003 · 2003
Earlier work this paper cites.
Fixing weight decay regularization in Adam
Ilya Loshchilov and Frank Hutter. 2017 · 2017
Earlier work this paper cites.
Rapid adaptation of neural machine translation to new languages
Graham Neubig and Junjie Hu. 2018 · 2018
Earlier work this paper cites.
CODAH: An adversarially-authored question answering dataset for common sense
Michael Chen, Mike D’Arcy, Alisa Liu, Jared Fernandez, and Doug Downey. 2019 · 2019
Earlier work this paper cites.
Cross-lingual language model pretraining
Alexis Conneau and Guillaume Lample. 2019 · 2019
Earlier work this paper cites.
Unsupervised domain adaptation of contextualized embeddings for sequence labeling
Xiaochuang Han and Jacob Eisenstein. 2019 · 2019
Earlier work this paper cites.
ZeRO: Memory optimizations toward training trillion parameter models
Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 2019 · 2019
Earlier work this paper cites.
CommonsenseQA: A question answering challenge targeting commonsense knowledge
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019 · 2019
Earlier work this paper cites.
Parsing with multilingual BERT, a small corpus, and a small treebank
Ethan C. Chau, Lucy H. Lin, and Noah A. Smith. 2020 · 2020
Earlier work this paper cites.
Emerging cross-lingual structure in pretrained language models
Alexis Conneau, Shijie Wu, Haoran Li, Luke Zettlemoyer, and Veselin Stoyanov. 2020 · 2020
Earlier work this paper cites.
Don’t stop pretraining: Adapt language models to domains and tasks
Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020 · 2020
Earlier work this paper cites.
The state and fate of linguistic diversity and inclusion in the NLP world
Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020 · 2020
Cited alongside, same era.
JAQKET: Constructing a Japanese QA dataset based on quiz questions [JAQKET:クイズを題材にした日本語QAデータセットの構築] (in Japanese)
Masatoshi Suzuki, Jun Suzuki, Koji Matsuda, Kyosuke Nishida, and Naoya Inoue. 2020 · 2020
Cited alongside, same era.
Extending multilingual BERT to low-resource languages
Zihan Wang, Karthikeyan K, Stephen Mayhew, and Dan Roth. 2020 · 2020
Cited alongside, same era.
nmT5 - is parallel data still relevant for pre-training massively multilingual language models?
Mihir Kale, Aditya Siddhant, Rami Al-Rfou, Linting Xue, Noah Constant, and Melvin Johnson. 2021 · 2021
Cited alongside, same era.
Common sense beyond English: Evaluating and improving multilingual language models for commonsense reasoning
Bill Yuchen Lin, Seyeon Lee, Xiaoyang Qiao, and Xiang Ren. 2021 · 2021
Cited alongside, same era.
Efficient and effective text encoding for Chinese LLaMA and Alpaca
Yiming Cui, Ziqing Yang, and Xin Yao. 2023 · 2023
Later among the works it cites.
FlashAttention-2: Faster attention with better parallelism and work partitioning
Tri Dao. 2023 · 2023
Later among the works it cites.
Do multilingual language models think better in english?
Julen Etxaniz, Gorka Azkune, Aitor Soroa Etxabe, Oier Lopez de Lacalle, and Mikel Artetxe. 2023 · 2023
Later among the works it cites.
Not all languages are created equal in LLMs: Improving multilingual capability by cross-lingual-thought prompting
Haoyang Huang, Tianyi Tang, Dongdong Zhang, Xin Zhao, Ting Song, Yan Xia, and Furu Wei. 2023 · 2023
Later among the works it cites.
Construction of a Japanese multi-hop QA dataset for a question-answering system that can explain its reasons [根拠を説明可能な質問応答システムのための日本語マルチホップQAデータセット構築] (in Japanese)
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
XLM-K: Improving cross-lingual language model pre-training with multilingual knowledge
Xiaoze Jiang, Yaobo Liang, Weizhu Chen, and Nan Duan. 2022 · 2022
Cited alongside, same era.
JGLUE: Japanese general language understanding evaluation
Kentaro Kurihara, Daisuke Kawahara, and Tomohide Shibata. 2022 · 2022
Cited alongside, same era.
Cedille: A large autoregressive French language model
Martin Müller and Florian Laurent. 2022 · 2022
Cited alongside, same era.
PARADISE: Exploiting parallel data for multilingual sequence-to-sequence pretraining
Machel Reid and Mikel Artetxe. 2022 · 2022
Cited alongside, same era.
mLUKE: The power of entity representations in multilingual pretrained language models
Ryokan Ri, Ikuya Yamada, and Yoshimasa Tsuruoka. 2022 · 2022
Cited alongside, same era.
Expanding pretrained models to thousands more languages via lexicon-based adaptation
Xinyi Wang, Sebastian Ruder, and Graham Neubig. 2022 · 2022
Cited alongside, same era.
MEGA: Multilingual evaluation of generative AI
Kabir Ahuja, Harshita Diddee, Rishav Hada, Millicent Ochieng, Krithika Ramesh, Prachi Jain, Akshay Nambi, Tanuja Ganu, Sameer Segal, Mohamed Ahmed, Kalika Bali, and Sunayana Sitaram. 2023 · 2023
Cited alongside, same era.
Ai Ishii, Naoya Inoue, and Satoshi Sekine. 2023 · 2023
Later among the works it cites.
SeaLLMs - large language models for Southeast Asia
Xuan-Phi Nguyen, Wenxuan Zhang, Xin Li, Mahani Aljunied, Qingyu Tan, Liying Cheng, Guanzheng Chen, Yue Deng, Sen Yang, Chaoqun Liu, Hang Zhang, and Li Bing. 2023 · 2023
Later among the works it cites.
Llama 2: Open foundation and fine-tuned chat models
Hugo Touvron, Louis Martin, Kevin R. Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Daniel M. Bikel, Lukas Blecher, Cristian Cantón Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony S. Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel M. Kloumann, A. V. Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, R. Subramanian, Xia Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zhengxu Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023 · 2023
Later among the works it cites.
BLOOM+1: Adding language support to BLOOM for zero-shot prompting
Zheng Xin Yong, Hailey Schoelkopf, Niklas Muennighoff, Alham Fikri Aji, David Ifeoluwa Adelani, Khalid Almubarak, M Saiful Bari, Lintang Sutawika, Jungo Kasai, Ahmed Baruwa, Genta Winata, Stella Biderman, Edward Raff, Dragomir Radev, and Vassilina Nikoulina. 2023 · 2023
Later among the works it cites.
Continual pre-training for cross-lingual LLM adaptation: Enhancing Japanese language capabilities
Kazuki Fujii, Taishi Nakamura, Mengsay Loem, Hiroki Iida, Masanari Ohi, Kakeru Hattori, Hirai Shota, Sakae Mizuki, Rio Yokota, and Naoaki Okazaki. 2024 · 2024
Closest in time.
Automatic evaluation tool for Japanese large language models [llm-jp-eval: 日本語大規模言語モデルの自動評価ツール] (in Japanese)
Namgi Han, Nobuhiro Ueda, Masatoshi Otake, Satoru Katsumata, Keisuke Kamata, Hirokazu Kiyomaru, Takashi Kodama, Saku Sugawara, Bowen Chen, Hiroshi Matsuda, Yusuke Miyao, Yugo Miyawaki, and Koki Ryu. 2024 · 2024
Closest in time.
LLaMA beyond English: An empirical study on language capability transfer
Jun Zhao, Zhihao Zhang, Qi Zhang, Tao Gui, and Xuanjing Huang. 2024 · 2024
Closest in time.