One Mind, Many Tongues: A Deep Dive into Language-Agnostic Knowledge Neurons in Large Language Models

J. McCarthy, “Some expert systems need common sense,” Annals of the New York Academy of Sciences , vol. 426, no. 1, pp. 129–137, 1984

1984

Earlier work this paper cites.

R. Davis, H. Shrobe, and P. Szolovits, “What is a knowledge representation?” AI magazine , vol. 14, no. 1, pp. 17–17, 1993

1993

Earlier work this paper cites.

J. Hyman, “How knowledge works,” The philosophical quarterly , vol. 49, no. 197, pp. 433–451, 1999

1999

Earlier work this paper cites.

G. Tononi, O. Sporns, and G. M. Edelman, “Measures of degeneracy and redundancy in biological networks,” Proceedings of the National Academy of Sciences , vol. 96, no. 6, pp. 3257–3262, 1999

1999

Earlier work this paper cites.

D. Baehrens, T. Schroeter, S. Harmeling, M. Kawanabe, K. Hansen, and K.-R. Müller, “How to explain individual classification decisions,” The Journal of Machine Learning Research , vol. 11, pp. 1803–1831, 2010

2010

Earlier work this paper cites.

K. Simonyan, A. Vedaldi, and A. Zisserman, “Deep inside convolutional networks: Visualising image classification models and saliency maps,” arXiv preprint arXiv:1312.6034 , 2013

Original

2013

Earlier work this paper cites.

P. H. Mason, “Degeneracy: Demystifying and destigmatizing a core concept in systems biology,” Complexity , vol. 20, no. 3, pp. 12–21, 2015

2015

Earlier work this paper cites.

M. Sundararajan, A. Taly, and Q. Yan, “Axiomatic attribution for deep networks,” in International conference on machine learning . PMLR, 2017, pp. 3319–3328

2017

Earlier work this paper cites.

H. Elsahar, P. Vougiouklis, A. Remaci, C. Gravier, J. Hare, F. Laforest, and E. Simperl, “T-rex: A large scale alignment of natural language with knowledge base triples,” in Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018) , 2018

2018

Earlier work this paper cites.

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805 , 2018

Original

2018

Earlier work this paper cites.

F. Petroni, T. Rocktäschel, S. Riedel, P. Lewis, A. Bakhtin, Y. Wu, and A. Miller, “Language models as knowledge bases?” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , 2019, pp. 2463–2473

2019

Earlier work this paper cites.

T. Pires, E. Schlinger, and D. Garrette, “How multilingual is multilingual bert?” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , 2019, pp. 4996–5001

2019

Earlier work this paper cites.

Z. Jiang, A. Anastasopoulos, J. Araki, H. Ding, and G. Neubig, “X-factr: Multilingual factual knowledge retrieval from pretrained language models,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , 2020, pp. 5943–5959

2020

Earlier work this paper cites.

Y. Elazar, N. Kassner, S. Ravfogel, A. Ravichander, E. Hovy, H. Schütze, and Y. Goldberg, “Measuring and improving consistency in pretrained language models,” Transactions of the Association for Computational Linguistics , vol. 9, pp. 1012–1031, 2021

2021

Earlier work this paper cites.

N. Kassner, P. Dufter, and H. Schütze, “Multilingual lama: Investigating knowledge in multilingual pretrained language models,” in Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume , 2021, pp. 3250–3258

2021

Earlier work this paper cites.

Z. Zhong, D. Friedman, and D. Chen, “Factual probing is [mask]: Learning vs. learning to recall,” in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , 2021, pp. 5017–5033

2021

Cited alongside, same era.

X. L. Li and P. Liang, “Prefix-tuning: Optimizing continuous prompts for generation,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , 2021, pp. 4582–4597

2021

Cited alongside, same era.

M. Geva, R. Schuster, J. Berant, and O. Levy, “Transformer feed-forward layers are key-value memories,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , 2021, pp. 5484–5495

2021

Cited alongside, same era.

S. Sanyal and X. Ren, “Discretized integrated gradients for explaining language models,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , 2021, pp. 10 285–10 299

G. Kamoda, B. Heinzerling, K. Sakaguchi, and K. Inui, “Test-time augmentation for factual probing,” in Findings of the Association for Computational Linguistics: EMNLP 2023 , 2023, pp. 3650–3661

2023

Later among the works it cites.

Z. Wang, L. Ye, H. Wang, W. C. Kwan, D. Ho, and K.-F. Wong, “Readprompt: A readable prompting method for reliable knowledge probing,” in Findings of the Association for Computational Linguistics: EMNLP 2023 , 2023, pp. 7468–7479

2023

Later among the works it cites.

X. Liu, Y. Zheng, Z. Du, M. Ding, Y. Qian, Z. Yang, and J. Tang, “Gpt understands, too,” AI Open , 2023

2023

Later among the works it cites.

M. Geva, J. Bastings, K. Filippova, and A. Globerson, “Dissecting recall of factual associations in auto-regressive language models,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , 2023, pp. 12 216–12 235

2023

Later among the works it cites.