Fetching the paper…
Reading the bibliography…
A standard practice when using large language models is for users to supplement their instruction with an input context containing new information for the model to process.
Realm: Retrieval-augmented language model pre-training, 2020
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang · 2002
Earlier work this paper cites.
Retrieval-augmented generation for knowledge-intensive nlp tasks, 2021
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela · 2005
Earlier work this paper cites.
An empirical investigation of catastrophic forgetting in gradient-based neural networks, 2015
Ian J. Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio · 2015
Earlier work this paper cites.
SQuAD: 100,000+ questions for machine comprehension of text
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang · 2016
Earlier work this paper cites.
spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing
Matthew Honnibal and Ines Montani · 2017
Earlier work this paper cites.
Measuring catastrophic forgetting in neural networks
Ronald Kemker, Angelina Abitino, Marc McClure, and Christopher Kanan · 2017
Earlier work this paper cites.
Think you have solved question answering? try arc, the ai2 reasoning challenge
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord · 2018
Earlier work this paper cites.
Training verifiers to solve math word problems
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman · 2021
Earlier work this paper cites.
Measuring massive multitask language understanding
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt · 2021
Earlier work this paper cites.
Hung-Ting Chen, Michael J. Q. Zhang, and Eunsol Choi · 2022
Earlier work this paper cites.
Large language models with controllable working memory, 2022
Daliang Li, Ankit Singh Rawat, Manzil Zaheer, Xin Wang, Michal Lukasik, Andreas Veit, Felix Yu, and Sanjiv Kumar · 2022
Earlier work this paper cites.
Entity-based knowledge conflicts in question answering, 2022
Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh · 2022
Earlier work this paper cites.
Ella Neeman, Roee Aharoni, Or Honovich, Leshem Choshen, Idan Szpektor, and Omri Abend · 2022
Earlier work this paper cites.
Two-stage llm fine-tuning with less specialization and more generalization
Yihan Wang, Si Si, Daliang Li, Michal Lukasik, Felix X. Yu, Cho-Jui Hsieh, Inderjit S. Dhillon, and Sanjiv Kumar · 2022
Cited alongside, same era.
Dissecting recall of factual associations in auto-regressive language models, 2023
Mor Geva, Jasmijn Bastings, Katja Filippova, and Amir Globerson · 2023
Cited alongside, same era.
An empirical study of catastrophic forgetting in large language models during continual fine-tuning
Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang · 2023
Cited alongside, same era.
Locating and editing factual associations in gpt, 2023
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov · 2023
Cited alongside, same era.
Detecting and mitigating hallucinations in multilingual summarisation, 2023
Understanding finetuning for factual knowledge extraction, 2024
Gaurav Ghosal, Tatsunori Hashimoto, and Aditi Raghunathan · 2024
Closest in time.
Tug-of-war between knowledge: Exploring and resolving knowledge conflicts in retrieval-augmented language models
Zhuoran Jin, Pengfei Cao, Yubo Chen, Kang Liu, Xiaojian Jiang, Jiexin Xu, Li Qiuxia, and Jun Zhao · 2024
Closest in time.
Cutting off the head ends the conflict: A mechanism for interpreting and mitigating knowledge conflicts in language models
Zhuoran Jin, Pengfei Cao, Hongbang Yuan, Yubo Chen, Jiexin Xu, Huaijun Li, Xiaojian Jiang, Kang Liu, and Jun Zhao · 2024
Closest in time.
Studying large language model behaviors under realistic knowledge conflicts, 2024
Evgenii Kortukov, Alexander Rubinstein, Elisa Nguyen, and Seong Joon Oh · 2024
Closest in time.
Understanding catastrophic forgetting in language models via implicit inference, 2024
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Yifu Qiu, Yftah Ziser, Anna Korhonen, Edoardo M. Ponti, and Shay B. Cohen · 2023
Cited alongside, same era.
Trusting your evidence: Hallucinate less with context-aware decoding, 2023
Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Scott Wen tau Yih · 2023
Cited alongside, same era.
Stanford alpaca: An instruction-following llama model
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto · 2023
Cited alongside, same era.
How far can camels go? exploring the state of instruction tuning on open resources, 2023
Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A. Smith, Iz Beltagy, and Hannaneh Hajishirzi · 2023
Cited alongside, same era.
Context-faithful prompting for large language models, 2023
Wenxuan Zhou, Sheng Zhang, Hoifung Poon, and Muhao Chen · 2023
Cited alongside, same era.
Evaluating correctness and faithfulness of instruction-following models for question answering, 2024
Vaibhav Adlakha, Parishad BehnamGhader, Xing Han Lu, Nicholas Meade, and Siva Reddy · 2024
Cited alongside, same era.
Lora learns less and forgets less, 2024
Dan Biderman, Jacob Portes, Jose Javier Gonzalez Ortiz, Mansheej Paul, Philip Greengard, Connor Jennings, Daniel King, Sam Havens, Vitaliy Chiley, Jonathan Frankle, Cody Blakeney, and John P. Cunningham · 2024
Cited alongside, same era.
Tianqing Fang, Zhaowei Wang, Wenxuan Zhou, Hongming Zhang, Yangqiu Song, and Muhao Chen · 2024
Cited alongside, same era.
Suhas Kotha, Jacob Mitchell Springer, and Aditi Raghunathan · 2024
Closest in time.
Inverse scaling: When bigger isn’t better, 2024
Ian R. McKenzie, Alexander Lyzhov, Michael Pieler, Alicia Parrish, Aaron Mueller, Ameya Prabhu, Euan McLean, Aaron Kirtland, Alexis Ross, Alisa Liu, Andrew Gritsevskiy, Daniel Wurgaft, Derik Kauffman, Gabriel Recchia, Jiacheng Liu, Joe Cavanagh, Max Weiss, Sicong Huang, The Floating Droid, Tom Tseng, Tomasz Korbak, Xudong Shen, Yuhui Zhang, Zhengping Zhou, Najoung Kim, Samuel R. Bowman, and Ethan Perez · 2024
Closest in time.
What does the knowledge neuron thesis have to do with knowledge?, 2024
Jingcheng Niu, Andrew Liu, Zining Zhu, and Gerald Penn · 2024
Closest in time.
Adacad: Adaptively decoding to balance conflicts between contextual and parametric knowledge, 2024
Han Wang, Archiki Prasad, Elias Stengel-Eskin, and Mohit Bansal · 2024
Closest in time.
Jian Xie, Kai Zhang, Jiangjie Chen, Renze Lou, and Yu Su · 2024
Closest in time.
Xiaowei Yuan, Zhao Yang, Yequan Wang, Shengping Liu, Jun Zhao, and Kang Liu · 2024
Closest in time.
Mitigating temporal misalignment by discarding outdated facts, 2024
Michael J. Q. Zhang and Eunsol Choi · 2024
Closest in time.
Raft: Adapting language model to domain specific rag, 2024
Tianjun Zhang, Shishir G. Patil, Naman Jain, Sheng Shen, Matei Zaharia, Ion Stoica, and Joseph E. Gonzalez · 2024
Closest in time.