Fetching the paper…
Reading the bibliography…
We ask whether multilingual language models trained on unbalanced, English-dominated corpora use English as an internal pivot language -- a question of key importance for understanding how language models function and the origins of linguistic bias.
Sex, syntax, and semantics
Lera Boroditsky, Lauren A. Schmidt, and Webb Phillips. 2003 · 2003
Earlier work this paper cites.
On the language neutrality of pre-trained multilingual representations
Jindřich Libovický, Rudolf Rosa, and Alexander Fraser. 2020 · 2004
Earlier work this paper cites.
With the future behind them: Convergent evidence from aymara language and gesture in the crosslinguistic comparison of spatial construals of time
Rafael E. Núñez and Eve Sweetser. 2006 · 2006
Earlier work this paper cites.
Rochelle Choenni and Ekaterina Shutova. 2020 · 2009
Earlier work this paper cites.
How transferable are features in deep neural networks?
Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. 2014 · 2014
Earlier work this paper cites.
Understanding neural networks through deep visualization
Jason Yosinski, Jeff Clune, Anh Nguyen, Thomas Fuchs, and Hod Lipson. 2015 · 2015
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018 · 2018
Earlier work this paper cites.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019 · 2019
Earlier work this paper cites.
Cross-lingual alignment of contextual word embeddings, with applications to zero-shot dependency parsing
Tal Schuster, Ori Ram, Regina Barzilay, and Amir Globerson. 2019 · 2019
Earlier work this paper cites.
On the cross-lingual transferability of monolingual representations
Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. 2020 · 2020
Earlier work this paper cites.
Multilingual alignment of contextual word representations
Steven Cao, Nikita Kitaev, and Dan Klein. 2020 · 2020
Earlier work this paper cites.
Multilingual denoising pre-training for neural machine translation
Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. 2020 · 2020
Earlier work this paper cites.
Interpreting gpt: The logit lens
Nostalgebraist. 2020 · 2020
Earlier work this paper cites.
Colexification networks encode affective meaning
Anna Di Natale, Max Pellert, and David Garcia. 2021 · 2021
Earlier work this paper cites.
A mathematical framework for transformer circuits
Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, et al. 2021 · 2021
Earlier work this paper cites.
First align, then predict: Understanding the cross-lingual ability of multilingual bert
Benjamin Muller, Yanai Elazar, Benoît Sagot, and Djamé Seddah. 2021 · 2021
Earlier work this paper cites.
mt5: A massively multilingual pre-trained text-to-text transformer
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021 · 2021
Earlier work this paper cites.
LLM.int8(): 8-bit matrix multiplication for transformers at scale
Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022 · 2022
Earlier work this paper cites.
Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space
Mor Geva, Avi Caciularu, Kevin Ro Wang, and Yoav Goldberg. 2022 · 2022
Cited alongside, same era.
Emergent world representations: Exploring a sequence model trained on a synthetic task
Kenneth Li, Aspen K Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. 2022 · 2022
Cited alongside, same era.
Few-shot learning with multilingual generative language models
Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O’Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, Mona Diab, Veselin Stoyanov, and Xian Li. 2022 · 2022
Cited alongside, same era.
Locating and editing factual associations in gpt
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022 · 2022
Cited alongside, same era.
Multilingual bert has an accent: Evaluating english influences on fluency in multilingual models
Samuel Marks and Max Tegmark. 2023 · 2023
Later among the works it cites.
A glitch in the matrix? locating and detecting language model grounding with fakepedia
Giovanni Monea, Maxime Peyrard, Martin Josifoski, Vishrav Chaudhary, Jason Eisner, Emre Kıcıman, Hamid Palangi, Barun Patra, and Robert West. 2023 · 2023
Later among the works it cites.
Progress measures for grokking via mechanistic interpretability
Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. 2023 · 2023
Later among the works it cites.
OpenAI. 2023 · 2023
Later among the works it cites.
Finland’s chatgpt equivalent begins to think in estonian as well
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Isabel Papadimitriou, Kezia Lopez, and Dan Jurafsky. 2022 · 2022
Cited alongside, same era.
Bloom: A 176b-parameter open-access multilingual language model
Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, et al. 2022 · 2022
Cited alongside, same era.
Language models are multilingual chain-of-thought reasoners
Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, Dipanjan Das, and Jason Wei. 2022 · 2022
Cited alongside, same era.
mgpt: Few-shot learners go multilingual
Oleh Shliazhko, Alena Fenogenova, Maria Tikhonova, Vladislav Mikhailov, Anastasia Kozlova, and Tatiana Shavrina. 2022 · 2022
Cited alongside, same era.
Chain-of-thought prompting elicits reasoning in large language models
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022 · 2022
Cited alongside, same era.
Mega: Multilingual evaluation of generative ai
Kabir Ahuja, Harshita Diddee, Rishav Hada, Millicent Ochieng, Krithika Ramesh, Prachi Jain, Akshay Nambi, Tanuja Ganu, Sameer Segal, Maxamed Axmed, Kalika Bali, and Sunayana Sitaram. 2023 · 2023
Cited alongside, same era.
Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, et al. 2023 · 2023
Cited alongside, same era.
Eliciting latent predictions from transformers with the tuned lens
Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman, and Jacob Steinhardt. 2023 · 2023
Cited alongside, same era.
Rait Piir. 2023 · 2023
Later among the works it cites.
Training dynamics of contextual n-grams in language models
Lucia Quirke, Lovis Heindrich, Wes Gurnee, and Neel Nanda. 2023 · 2023
Later among the works it cites.
Decoding intermediate activations in Llama-2-7b
Nina Rimsky. 2023 · 2023
Later among the works it cites.
Llama 2: Open foundation and fine-tuned chat models
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023 · 2023
Later among the works it cites.
Polylm: An open source polyglot large language model
Xiangpeng Wei, Haoran Wei, Huan Lin, Tianhao Li, Pei Zhang, Xingzhang Ren, Mei Li, Yu Wan, Zhiwei Cao, Binbin Xie, Tianxiang Hu, Shangjie Li, Binyuan Hui, Bowen Yu, Dayiheng Liu, Baosong Yang, Fei Huang, and Jun Xie. 2023 · 2023
Later among the works it cites.
Don’t trust ChatGPT when your question is not in English: A study of multilingual abilities and types of LLMs
Xiang Zhang, Senyu Li, Bradley Hauer, Ning Shi, and Grzegorz Kondrak. 2023 · 2023
Later among the works it cites.
Extrapolating large language models to non-english by aligning languages
Wenhao Zhu, Yunzhe Lv, Qingxiu Dong, Fei Yuan, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, and Lei Li. 2023 · 2023
Later among the works it cites.
Spectral filters, dark signals, and attention sinks
Nicola Cancedda. 2024 · 2024
Closest in time.
Llama-polyglot-13b
Charles Goddard. 2023 · 2024
Closest in time.
vigogne-2-13b-instruct
Bofeng Huang. 2023 · 2024
Closest in time.
Jaavid Aktar Husain, Raj Dabre, Aswanth Kumar, Ratish Puduppully, and Anoop Kunchukuttan. 2024 · 2024
Closest in time.
Llama-2-ko-dpo-13b
Daekeun Kim. 2023 · 2024
Closest in time.
LeoLM: Ein Impuls für Deutschsprachige LLM-Forschung
Björn Plüster. 2023 · 2024
Closest in time.
Question translation training for better multilingual reasoning
Wenhao Zhu, Shujian Huang, Fei Yuan, Shuaijie She, Jiajun Chen, and Alexandra Birch. 2024 · 2024
Closest in time.