Fetching the paper…

Reading the bibliography…

2024

EVEDIT: Event-based Knowledge Editing with Deductive Editing Boundaries

Liu, Jiateng, Yu, Pengfei, Zhang, Yuji et al.

Original
PDF
Post on X

Understand

The dynamic nature of real-world information necessitates efficient knowledge editing (KE) in large language models (LLMs) for knowledge updating.

However, current KE approaches, which typically operate on (subject, relation, object) triples, ignore the contextual information and the relation among different knowledge.
Such editing methods could thus encounter an uncertain editing boundary, leaving a lot of relevant knowledge in ambiguity: Queries that could be answered pre-edit cannot be reliably answered afterward.
In this work, we analyze this issue by introducing a theoretical framework for KE that highlights an overlooked set of knowledge that remains unchanged and aids in knowledge deduction during editing, which we name as the deduction anchor.

EVEDIT: Event-based Knowledge Editing with Deductive Editing Boundaries · Around

Built on

Generalization through memorization: Nearest neighbor language models
Original
Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2019 · 1911
Earlier work this paper cites.
An introduction to formal logic
Peter Smith. 2003 · 2003
Earlier work this paper cites.
Zero-shot relation extraction via reading comprehension
Original
Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. 2017 · 2017
Earlier work this paper cites.
Realm: Retrieval-augmented language model pre-training
Original
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020 · 2020
Earlier work this paper cites.
Editing factual knowledge in language models
Original
Nicola De Cao, Wilker Aziz, and Ivan Titov. 2021 · 2021
Earlier work this paper cites.
Lora: Low-rank adaptation of large language models
Original
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021 · 2021
Earlier work this paper cites.
Fast model editing at scale
Original
Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D Manning. 2021 · 2021
Earlier work this paper cites.
Improving language models by retrieving from trillions of tokens
Original
Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George van den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, Diego de Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, Geoffrey Irving, Oriol Vinyals, Simon Osindero, Karen Simonyan, Jack W. Rae, Erich Elsen, and Laurent Sifre. 2022 · 2022
Earlier work this paper cites.
Calibrating factual knowledge in pretrained language models
Original
Qingxiu Dong, Damai Dai, Yifan Song, Jingjing Xu, Zhifang Sui, and Lei Li. 2022 · 2022
Earlier work this paper cites.
Memory-based model editing at scale
Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D Manning, and Chelsea Finn. 2022 · 2022
Earlier work this paper cites.
Fixing model bugs with natural language patches
Original
Shikhar Murty, Christopher D. Manning, Scott Lundberg, and Marco Tulio Ribeiro. 2022 · 2022
Earlier work this paper cites.

Similar

Recitation-augmented language models
Original
Zhiqing Sun, Xuezhi Wang, Yi Tay, Yiming Yang, and Denny Zhou. 2022 · 2022
Cited alongside, same era.
Towards a unified multi-dimensional evaluator for text generation
Original
Ming Zhong, Yang Liu, Da Yin, Yuning Mao, Yizhu Jiao, Pengfei Liu, Chenguang Zhu, Heng Ji, and Jiawei Han. 2022 · 2022
Cited alongside, same era.
Self-rag: Learning to retrieve, generate, and critique through self-reflection
Original
Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2023 · 2023
Cited alongside, same era.
Evaluating the ripple effects of knowledge editing in language models
Original
Roi Cohen, Eden Biran, Ori Yoran, Amir Globerson, and Mor Geva. 2023 · 2023
Cited alongside, same era.
Aging with grace: Lifelong model editing with discrete key-value adaptors

Then

Toolformer: Language models can teach themselves to use tools
Original
Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023 · 2023
Later among the works it cites.
Vipergpt: Visual inference via python execution for reasoning
Original
Dídac Surís, Sachit Menon, and Carl Vondrick. 2023 · 2023
Later among the works it cites.
Massive editing for large language models via meta learning
Original
Chenmien Tan, Ge Zhang, and Jie Fu. 2023 · 2023
Later among the works it cites.
Editing large language models: Problems, methods, and opportunities
Original
Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. 2023 · 2023
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Original

Thomas Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim, and Marzyeh Ghassemi. 2023 · 2023

Cited alongside, same era.

Inspecting and editing knowledge representations in language models

Evan Hernandez, Belinda Z. Li, and Jacob Andreas. 2023 · 2023

Cited alongside, same era.

Transformer-patcher: One mistake worth one neuron

Zeyu Huang, Yikang Shen, Xiaofeng Zhang, Jie Zhou, Wenge Rong, and Zhang Xiong. 2023 · 2023

Cited alongside, same era.

Knowledge graph-augmented language models for knowledge-grounded dialogue generation

Original

Minki Kang, Jin Myung Kwak, Jinheon Baek, and Sung Ju Hwang. 2023 · 2023

Cited alongside, same era.

Pmet: Precise model editing in a transformer

Xiaopeng Li, Shasha Li, Shezheng Song, Jing Yang, Jun Ma, and Jie Yu. 2023 · 2023

Cited alongside, same era.

A language-first approach for procedure planning

Jiateng Liu, Sha Li, Zhenhailong Wang, Manling Li, and Heng Ji. 2023 · 2023

Cited alongside, same era.

Joint multimedia event extraction from video and article

Brian Chen, Xudong Lin, Christopher Thomas, Manling Li, Shoya Yoshida, Lovish Chum, Heng Ji, and Shih-Fu Chang. 2021a

Cited in the paper.

Event-centric natural language processing

Muhao Chen, Hongming Zhang, Qiang Ning, Manling Li, Heng Ji, Kathleen McKeown, and Dan Roth. 2021b

Cited in the paper.

Melo: Enhancing model editing with neuron-indexed dynamic lora

Lang Yu, Qin Chen, Jie Zhou, and Liang He. 2023 · 2023

Later among the works it cites.

Self information update for large language models through mitigating exposure bias

Pengfei Yu and Heng Ji. 2023 · 2023

Later among the works it cites.

VIBE: Topic-driven temporal adaptation for Twitter classification

Yuji Zhang, Jing Li, and Wenjie Li. 2023b · 2023

Later among the works it cites.

Retrieval-augmented generation for large language models: A survey

Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Qianyu Guo, Meng Wang, and Haofen Wang. 2024 · 2024

Closest in time.

If llm is the wizard, then code is the wand: A survey on how code empowers large language models to serve as intelligent agents

Ke Yang, Jiateng Liu, John Wu, Chaoqi Yang, Yi R. Fung, Sha Li, Zixuan Huang, Xu Cao, Xingyao Wang, Yiquan Wang, Heng Ji, and Chengxiang Zhai. 2024 · 2024

Closest in time.

A comprehensive study of knowledge editing for large language models

Ningyu Zhang, Yunzhi Yao, Bozhong Tian, Peng Wang, Shumin Deng, Mengru Wang, Zekun Xi, Shengyu Mao, Jintian Zhang, Yuansheng Ni, Siyuan Cheng, Ziwen Xu, Xin Xu, Jia-Chen Gu, Yong Jiang, Pengjun Xie, Fei Huang, Lei Liang, Zhiqiang Zhang, Xiaowei Zhu, Jun Zhou, and Huajun Chen. 2024 · 2024

Closest in time.