Fetching the paper…
Reading the bibliography…
As real-world knowledge evolves, the information embedded within large language models (LLMs) can become outdated, inadequate, or erroneous.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 1901
Earlier work this paper cites.
A learning algorithm for continually running fully recurrent neural networks
Ronald J Williams and David Zipser. 1989 · 1989
Earlier work this paper cites.
Zero-shot relation extraction via reading comprehension. In 21st Conference on Computational Natural Language Learning, CoNLL 2017 . Association for Computational Linguistics (ACL), 333–342
Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. 2017 · 2017
Earlier work this paper cites.
Language Models as Knowledge Bases?. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) . 2463–2473
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019 · 2019
Earlier work this paper cites.
How Can We Know What Language Models Know?
Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham Neubig. 2020 · 2020
Earlier work this paper cites.
Retrieval-augmented generation for knowledge-intensive nlp tasks
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al · 2020
Earlier work this paper cites.
Modifying memories in transformer models. In International Conference on Machine Learning (ICML) , Vol. 2020
Ankit Singh Rawat, Chen Zhu, Daliang Li, Felix Yu, Manzil Zaheer, Sanjiv Kumar, and Srinadh Bhojanapalli. 2021 · 2020
Earlier work this paper cites.
Anton Sinitsin, Vsevolod Plokhotnyuk, Dmitriy Pyrkin, Sergei Popov, and Artem Babenko. 2020 · 2020
Earlier work this paper cites.
Editing Factual Knowledge in Language Models. In EMNLP 2021-2021 Conference on Empirical Methods in Natural Language Processing, Proceedings . 6491–6506
N De Cao, W Aziz, and I Titov. 2021 · 2021
Earlier work this paper cites.
Transformer Feed-Forward Layers Are Key-Value Memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing . 5484–5495
Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021 · 2021
Earlier work this paper cites.
A review on language models as knowledge bases
Badr AlKhamissi, Millicent Li, Asli Celikyilmaz, Mona Diab, and Marjan Ghazvininejad. 2022 · 2022
Earlier work this paper cites.
Towards understanding the mixture-of-experts layer in deep learning. In Proceedings of the 36th International Conference on Neural Information Processing Systems . 23049–23062
Zixiang Chen, Yihe Deng, Yue Wu, Quanquan Gu, and Yuanzhi Li. 2022 · 2022
Earlier work this paper cites.
Knowledge Neurons in Pretrained Transformers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . 8493–8502
Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022 · 2022
Earlier work this paper cites.
Calibrating Factual Knowledge in Pretrained Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2022 . 5937–5947
Qingxiu Dong, Damai Dai, Yifan Song, Jingjing Xu, Zhifang Sui, and Lei Li. 2022 · 2022
Earlier work this paper cites.
Locating and editing factual associations in GPT
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022b · 2022
Earlier work this paper cites.
Memory-based model editing at scale. In International Conference on Machine Learning . PMLR, 15817–15831
Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D Manning, and Chelsea Finn. 2022 · 2022
Earlier work this paper cites.
Training language models to follow instructions with human feedback
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al · 2022
Earlier work this paper cites.
Emergent Abilities of Large Language Models
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al · 2022
Earlier work this paper cites.
Chain-of-thought prompting elicits reasoning in large language models
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al · 2022
Earlier work this paper cites.
Parameter-efficient fine-tuning of large-scale pre-trained language models
Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang, Yusheng Su, Shengding Hu, Yulin Chen, Chi-Min Chan, Weize Chen, et al · 2023
Earlier work this paper cites.
Dissecting Recall of Factual Associations in Auto-Regressive Language Models. In The 2023 Conference on Empirical Methods in Natural Language Processing
Mor Geva, Jasmijn Bastings, Katja Filippova, and Amir Globerson. [n. d.] · 2023
Earlier work this paper cites.
Inspecting and editing knowledge representations in language models
Evan Hernandez, Belinda Z Li, and Jacob Andreas. 2023 · 2023
Earlier work this paper cites.
Transformer-Patcher: One Mistake Worth One Neuron. In The Eleventh International Conference on Learning Representations
Zeyu Huang, Yikang Shen, Xiaofeng Zhang, Jie Zhou, Wenge Rong, and Zhang Xiong. 2023 · 2023
Earlier work this paper cites.
Survey of hallucination in natural language generation
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023 · 2023
Earlier work this paper cites.
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al · 2023
Cited alongside, same era.
Evaluating Open-Domain Question Answering in the Era of Large Language Models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . 5591–5606
Ehsan Kamalloo, Nouha Dziri, Charles Clarke, and Davood Rafiei. 2023 · 2023
Cited alongside, same era.
Transformers as algorithms: Generalization and stability in in-context learning. In International Conference on Machine Learning . PMLR, 19565–19594
Yingcong Li, Muhammed Emrullah Ildiz, Dimitris Papailiopoulos, and Samet Oymak. 2023 · 2023
Cited alongside, same era.
Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023 · 2023
Cited alongside, same era.
“Flex Tape Can’t Fix That”: Bias and Misinformation in Edited Language Models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing . 8690–8707
Karina Halevy, Anna Sotnikova, Badr Alkhamissi, Syrielle Montariol, and Antoine Bosselut. 2024 · 2024
Later among the works it cites.
Aging with grace: Lifelong model editing with discrete key-value adaptors
Tom Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim, and Marzyeh Ghassemi. 2024 · 2024
Later among the works it cites.
WilKE: Wise-Layer Knowledge Editor for Lifelong Knowledge Editing. In Findings of the Association for Computational Linguistics ACL 2024 . 3476–3503
Chenhui Hu, Pengfei Cao, Yubo Chen, Kang Liu, and Jun Zhao. 2024 · 2024
Later among the works it cites.
Learning to Edit: Aligning LLMs with Knowledge Editing. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . 4689–4705
Yuxin Jiang, Yufei Wang, Chuhan Wu, Wanjun Zhong, Xingshan Zeng, Jiahui Gao, Liangyou Li, Xin Jiang, Lifeng Shang, Ruiming Tang, et al · 2024
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. 2023 · 2023
Cited alongside, same era.
R Thomas McCoy, Shunyu Yao, Dan Friedman, Matthew Hardy, and Thomas L Griffiths. 2023 · 2023
Cited alongside, same era.
OpenAI. 2023 · 2023
Cited alongside, same era.
Emptying the Ocean with a Spoon: Should We Edit Models?. In Findings of the Association for Computational Linguistics: EMNLP 2023 . 15164–15172
Yuval Pinter and Michael Elhadad. 2023 · 2023
Cited alongside, same era.
Llama 2: Open foundation and fine-tuned chat models
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al · 2023
Cited alongside, same era.
Self-Instruct: Aligning Language Models with Self-Generated Instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . 13484–13508
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023 · 2023
Cited alongside, same era.
Self-Adaptive In-Context Learning: An Information Compression Perspective for In-Context Example Selection and Ordering. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . 1423–1436
Zhiyong Wu, Yaoxiang Wang, Jiacheng Ye, and Lingpeng Kong. 2023 · 2023
Cited alongside, same era.
Editing Large Language Models: Problems, Methods, and Opportunities. In The 2023 Conference on Empirical Methods in Natural Language Processing
Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. [n. d.] · 2023
Cited alongside, same era.
Shuaiyi Li, Yang Deng, Deng Cai, Hongyuan Lu, Liang Chen, and Wai Lam. 2024a · 2024
Later among the works it cites.
Interpreting Key Mechanisms of Factual Recall in Transformer-Based Language Models
Ang Lv, Kaiyi Zhang, Yuhan Chen, Yulong Wang, Lifeng Liu, Ji-Rong Wen, Jian Xie, and Rui Yan. 2024b · 2024
Later among the works it cites.
Full Parameter Fine-tuning for Large Language Models with Limited Resources. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 8187–8198
Kai Lv, Yuqing Yang, Tengxiao Liu, Qipeng Guo, and Xipeng Qiu. 2024a · 2024
Later among the works it cites.
Is Your LLM Outdated? Benchmarking LLMs & Alignment Algorithms for Time-Sensitive Knowledge
Seyed Mahed Mousavi, Simone Alghisi, and Giuseppe Riccardi. 2024 · 2024
Later among the works it cites.
UnUnlearning: Unlearning is not sufficient for content regulation in advanced generative AI
Ilia Shumailov, Jamie Hayes, Eleni Triantafillou, Guillermo Ortiz-Jimenez, Nicolas Papernot, Matthew Jagielski, Itay Yona, Heidi Howard, and Eugene Bagdasaryan. 2024 · 2024
Later among the works it cites.
RoseLoRA: Row and Column-wise Sparse Low-rank Adaptation of Pre-trained Language Model for Knowledge Editing and Fine-tuning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing . 996–1008
Haoyu Wang, Tianci Liu, Ruirui Li, Monica Cheng, Tuo Zhao, and Jing Gao. 2024c · 2024
Later among the works it cites.
The missing piece in model editing: A deep dive into the hidden damage brought by model editing
Jianchen Wang, Zhouhong Gu, Zhuozhi Xiong, Hongwei Feng, and Yanghua Xiao. 2024a · 2024
Later among the works it cites.
LEMoE: Advanced Mixture of Experts Adaptor for Lifelong Model Editing of Large Language Models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing . 2551–2575
Renzhi Wang and Piji Li. 2024a · 2024
Later among the works it cites.
MEMoE: Enhancing Model Editing with Mixture of Experts Adaptors
Renzhi Wang and Piji Li. 2024b · 2024
Later among the works it cites.
Knowledge editing for large language models: A survey
Song Wang, Yaochen Zhu, Haochen Liu, Zaiyi Zheng, Chen Chen, and Jundong Li. 2024d · 2024
Later among the works it cites.
Does knowledge localization hold true? surprising differences between entity and relation perspectives in language models. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management . 4118–4122
Yifan Wei, Xiaoyan Yu, Yixuan Weng, Huanhuan Ma, Yuanzhe Zhang, Jun Zhao, and Kang Liu. 2024 · 2024
Later among the works it cites.
Harnessing the power of llms in practice: A survey on chatgpt and beyond
Jingfeng Yang, Hongye Jin, Ruixiang Tang, Xiaotian Han, Qizhang Feng, Haoming Jiang, Shaochen Zhong, Bing Yin, and Xia Hu. 2024a · 2024
Later among the works it cites.
The Butterfly Effect of Model Editing: Few Edits Can Trigger Large Language Models Collapse. In Findings of the Association for Computational Linguistics ACL 2024 . 5419–5437
Wanli Yang, Fei Sun, Xinyu Ma, Xun Liu, Dawei Yin, and Xueqi Cheng. 2024b · 2024
Later among the works it cites.
The fall of rome: Understanding the collapse of llms in model editing. In Findings of the Association for Computational Linguistics: EMNLP 2024 . 4079–4087
Wanli Yang, Fei Sun, Jiajun Tan, Xinyu Ma, Du Su, Dawei Yin, and Huawei Shen. 2024c · 2024
Later among the works it cites.
Melo: Enhancing model editing with neuron-indexed dynamic lora. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 38. 19449–19457
Lang Yu, Qin Chen, Jie Zhou, and Liang He. 2024 · 2024
Later among the works it cites.
Knowledge Graph Enhanced Large Language Model Editing. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing . 22647–22662
Mengqi Zhang, Xiaotian Ye, Qiang Liu, Pengjie Ren, Shu Wu, and Zhumin Chen. 2024c · 2024
Later among the works it cites.
A comprehensive study of knowledge editing for large language models
Ningyu Zhang, Yunzhi Yao, Bozhong Tian, Peng Wang, Shumin Deng, Mengru Wang, Zekun Xi, Shengyu Mao, Jintian Zhang, Yuansheng Ni, et al · 2024
Later among the works it cites.
DAFNet: Dynamic Auxiliary Fusion for Sequential Model Editing in Large Language Models. In Findings of the Association for Computational Linguistics ACL 2024 . 1588–1602
Taolin Zhang, Qizhou Chen, Dongyang Li, Chengyu Wang, Xiaofeng He, Longtao Huang, Jun Huang, et al · 2024
Later among the works it cites.
Reft: Representation finetuning for language models
Zhengxuan Wu, Aryaman Arora, Zheng Wang, Atticus Geiger, Dan Jurafsky, Christopher D Manning, and Christopher Potts. 2025 · 2025
Closest in time.