Fetching the paper…
Reading the bibliography…
Current LLM benchmarks focus on evaluating models' memory of facts and semantic relations, primarily assessing semantic aspects of long-term memory.
The Murder of Roger Ackroyd
Agatha Christie · 1927
Earlier work this paper cites.
Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory
James L McClelland, Bruce L McNaughton, and Randall C O’Reilly · 1995
Earlier work this paper cites.
Separate mechanisms for short-and long-term memory
Iván Izquierdo, Jorge H Medina, Mônica RM Vianna, Luciana A Izquierdo, and Daniela M Barros · 1999
Earlier work this paper cites.
A distributed representation of temporal context
Marc W. Howard and Michael J. Kahana · 2001
Earlier work this paper cites.
A context maintenance and retrieval model of organizational processes in free recall
Sean M. Polyn, Kenneth A. Norman, and Michael J. Kahana · 2009
Earlier work this paper cites.
Memory on time
Howard Eichenbaum · 2013
Earlier work this paper cites.
How the hippocampus preserves order: the role of prediction and context
Lila Davachi and Sarah DuBrow · 2015
Earlier work this paper cites.
Hierarchical process memory: memory as an integral component of information processing
Uri Hasson, Janice Chen, and Christopher J Honey · 2015
Earlier work this paper cites.
Short-term memory and long-term memory are still different
Dennis Norris · 2017
Earlier work this paper cites.
T-rex: A large scale alignment of natural language with knowledge base triples
Hady Elsahar, Pavlos Vougiouklis, Arslen Remaci, Christophe Gravier, Jonathon Hare, Frederique Laforest, and Elena Simperl · 2018
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2019
Earlier work this paper cites.
Barack’s wife hillary: Using knowledge-graphs for fact-aware language modeling
Robert L Logan IV, Nelson F Liu, Matthew E Peters, Matt Gardner, and Sameer Singh · 2019
Earlier work this paper cites.
Language models as knowledge bases?
Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H Miller, and Sebastian Riedel · 2019
Earlier work this paper cites.
Measuring massive multitask language understanding
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt · 2020
Earlier work this paper cites.
Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer · 2020
Earlier work this paper cites.
Torque: A reading comprehension dataset of temporal ordering questions
Qiang Ning, Hao Wu, Rujun Han, Nanyun Peng, Matt Gardner, and Dan Roth · 2020
Earlier work this paper cites.
Huggingface’s transformers: State-of-the-art natural language processing, 2020
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush · 2020
Earlier work this paper cites.
The mechanisms underlying interference and inhibition: A review of current behavioral and neuroimaging research
Oliver Kliegl and Karl-Heinz T Bäuml · 2021
Earlier work this paper cites.
Long range arena : A benchmark for efficient transformers
Yi Tay, Mostafa Dehghani, Samira Abnar, Yikang Shen, Dara Bahri, Philip Pham, Jinfeng Rao, Liu Yang, Sebastian Ruder, and Donald Metzler · 2021
Earlier work this paper cites.
Temporal reasoning on implicit events from distant supervision
Ben Zhou, Kyle Richardson, Qiang Ning, Tushar Khot, Ashish Sabharwal, and Dan Roth · 2021
Earlier work this paper cites.
Improving language models by retrieving from trillions of tokens
Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et al · 2022
Earlier work this paper cites.
Muld: The multitask long document benchmark
George Hudson and Noura Al Moubayed · 2022
Earlier work this paper cites.
Scrolls: Standardized comparison over long language sequences
Uri Shaham, Elad Segal, Maor Ivgi, Avia Efrat, Ori Yoran, Adi Haviv, Ankit Gupta, Wenhan Xiong, Mor Geva, Jonathan Berant, et al · 2022
Cited alongside, same era.
ChapterBreak: A challenge dataset for long-range language models
Simeng Sun, Katherine Thai, and Mohit Iyyer · 2022
Cited alongside, same era.
Relating transformers to models and neural representations of the hippocampal formation
James C. R. Whittington, Joseph Warren, and Tim E.J. Behrens · 2022
Cited alongside, same era.
Training language models with memory augmentation
Zexuan Zhong, Tao Lei, and Danqi Chen · 2022
Cited alongside, same era.
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al · 2023
Cited alongside, same era.
Llama 3 model card
AI@Meta · 2024
Closest in time.
Larimar: Large language models with episodic memory control, 2024
Payel Das, Subhajit Chaudhury, Elliot Nelson, Igor Melnyk, Sarath Swaminathan, Sihui Dai, Aurélie Lozano, Georgios Kollias, Vijil Chenthamarakshan, Jiří, Navrátil, Soham Dan, and Pin-Yu Chen · 2024
Closest in time.
The Faiss library, 2024
Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou · 2024
Closest in time.
Short-term hebbian learning can implement transformer-like attention
Ian T. Ellwood · 2024
Closest in time.
Human-like episodic memory for infinite context llms, 2024
Zafeirios Fountas, Martin A Benfeghoul, Adnan Oomerjee, Fenia Christopoulou, Gerasimos Lampouras, Haitham Bou-Ammar, and Jun Wang · 2024
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Generic temporal reasoning with differential analysis and explanation
Yu Feng, Ben Zhou, Haoyu Wang, Helen Jin, and Dan Roth · 2023
Cited alongside, same era.
A practical survey on faster and lighter transformers
Quentin Fournier, Gaétan Marceau Caron, and Daniel Aloise · 2023
Cited alongside, same era.
Crow: Benchmarking commonsense reasoning in real-world tasks
Mete Ismayilzada, Debjit Paul, Syrielle Montariol, Mor Geva, and Antoine Bosselut · 2023
Cited alongside, same era.
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al · 2023
Cited alongside, same era.
Efficient memory management for large language model serving with pagedattention, 2023
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica · 2023
Cited alongside, same era.
Think-in-memory: Recalling and post-thinking enable llms with long-term memory
Lei Liu, Xiaoyan Yang, Yue Shen, Binbin Hu, Zhiqiang Zhang, Jinjie Gu, and Guannan Zhang · 2023
Cited alongside, same era.
RWKV: Reinventing RNNs for the transformer era
Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Leon Derczynski, Xingjian Du, Matteo Grella, Kranthi Gv, Xuzheng He, Haowen Hou, Przemyslaw Kazienko, Jan Kocon, Jiaming Kong, Bartłomiej Koptyra, Hayden Lau, Jiaju Lin, Krishna Sri Ipsit Mantri, Ferdinand Mom, Atsushi Saito, Guangyu Song, Xiangru Tang, Johan Wind, Stanisław Woźniak, Zhenyuan Zhang, Qinghua Zhou, Jian Zhu, and Rui-Jie Zhu · 2023
Cited alongside, same era.
Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang · 2024
Closest in time.
RULER: What’s the real context size of your long-context language models?
Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, and Boris Ginsburg · 2024
Closest in time.
Linking in-context learning in transformers to human episodic memory, 2024
Li Ji-An, Corey Y. Zhou, Marcus K. Benna, and Marcelo G. Mattar · 2024
Closest in time.
Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al · 2024
Closest in time.
Llmtest_needleinahaystack, 2023
Greg Kamradt · 2024
Closest in time.
Mosh Levy, Alon Jacoby, and Yoav Goldberg · 2024
Closest in time.
Lost in the middle: How language models use long contexts
Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang · 2024
Closest in time.
Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang · 2024
Closest in time.
Computational Cognitive Neuroscience
Randall C. O’Reilly, Yuko Munakata, Michael J. Frank, Thomas E. Hazy, and Contributors · 2024
Closest in time.
Gemma: Open models based on gemini research and technology
Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, et al · 2024
Closest in time.
Augmenting language models with long-term memory
Weizhi Wang, Li Dong, Hao Cheng, Xiaodong Liu, Xifeng Yan, Jianfeng Gao, and Furu Wei · 2024
Closest in time.
C-Pack: Packaged Resources To Advance General Chinese Embedding
Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, Defu Lian, and Jian-Yun Nie · 2024
Closest in time.
In defense of rag in the era of long-context language models, 2024
Tan Yu, Anbang Xu, and Rama Akkiraju · 2024
Closest in time.
Removing rlhf protections in gpt-4 via fine-tuning, 2024
Qiusi Zhan, Richard Fang, Rohan Bindu, Akul Gupta, Tatsunori Hashimoto, and Daniel Kang · 2024
Closest in time.
Instruction tuning for large language models: A survey, 2024
Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, and Guoyin Wang · 2024
Closest in time.
Memorybank: Enhancing large language models with long-term memory
Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang · 2024
Closest in time.