Fetching the paper…
Reading the bibliography…
In this paper, we introduce Writing in the Margins (WiM), a new inference pattern for Large Language Models designed to optimize the handling of long input sequences in retrieval-oriented tasks.
“Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context”, 2019
Zihang Dai et al · 1901
Earlier work this paper cites.
“Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context”, 2019
Zihang Dai et al · 1901
Earlier work this paper cites.
“Generalization through Memorization: Nearest Neighbor Language Models”, 2020
Urvashi Khandelwal et al · 1911
Earlier work this paper cites.
“Forgetting Exceptions is Harmful in Language Learning”, 1998
Walter Daelemans, Antal van Bosch and Jakub Zavrel · 1998
Earlier work this paper cites.
“Entities as Experts: Sparse Memory Access with Entity Supervision”, 2020
Thibault Févry et al · 2004
Earlier work this paper cites.
“Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks”, 2021
Patrick Lewis et al · 2005
Earlier work this paper cites.
“INT: An Inequality Benchmark for Evaluating Generalization in Theorem Proving”, 2021
Yuhuai Wu, Albert Jiang, Jimmy Ba and Roger Grosse · 2007
Earlier work this paper cites.
“Natural language processing with Python: analyzing text with the natural language toolkit”
Steven Bird, Ewan Klein and Edward Loper · 2009
Earlier work this paper cites.
“Learning Associative Inference Using Fast Weight Memory”, 2021
Imanol Schlag, Tsendsuren Munkhdalai and Jürgen Schmidhuber · 2011
Earlier work this paper cites.
“Know What You Don’t Know: Unanswerable Questions for SQuAD”, 2018
Pranav Rajpurkar, Robin Jia and Percy Liang · 2018
Earlier work this paper cites.
“HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering”, 2018
Zhilin Yang et al · 2018
Earlier work this paper cites.
“Leveraging passage retrieval with generative models for open domain question answering”
Gautier Izacard and Edouard Grave · 2020
Earlier work this paper cites.
“Program Synthesis with Large Language Models”, 2021
Jacob Austin et al · 2021
Earlier work this paper cites.
“Evaluating Large Language Models Trained on Code”, 2021
Mark Chen et al · 2021
Earlier work this paper cites.
“Show Your Work: Scratchpads for Intermediate Computation with Language Models”, 2021
Maxwell Nye et al · 2021
Earlier work this paper cites.
“Exploring Length Generalization in Large Language Models”, 2022
Cem Anil et al · 2022
Earlier work this paper cites.
“LangChain”, https://github.com/langchain-ai/langchain , 2022
Harrison Chase · 2022
Cited alongside, same era.
“Relational Memory Augmented Language Models”, 2022
Qi Liu, Dani Yogatama and Phil Blunsom · 2022
Cited alongside, same era.
“Parallel context windows improve in-context learning of large language models”
Nir Ratner et al · 2022
Cited alongside, same era.
“SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills”, 2023
Amey Agrawal et al · 2023
Cited alongside, same era.
“Efficient long-text understanding with short-text models”
Maor Ivgi, Uri Shaham and Jonathan Berant · 2023
Cited alongside, same era.
“Chain-of-Thought Prompting Elicits Reasoning in Large Language Models”, 2023
Jason Wei et al · 2023
Later among the works it cites.
“Chain-of-Thought Prompting Elicits Reasoning in Large Language Models”, 2023
Jason Wei et al · 2023
Later among the works it cites.
“Tree of Thoughts: Deliberate Problem Solving with Large Language Models”, 2023
Shunyu Yao et al · 2023
Later among the works it cites.
“What Algorithms can Transformers Learn? A Study in Length Generalization”, 2023
Hattie Zhou et al · 2023
Later among the works it cites.
“Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone”, 2024
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Woosuk Kwon et al · 2023
Cited alongside, same era.
“LooGLE: Can Long-Context Language Models Understand Long Contexts?”, 2023
Jiaqi Li, Mengmeng Wang, Zilong Zheng and Muhan Zhang · 2023
Cited alongside, same era.
“Lost in the Middle: How Language Models Use Long Contexts”, 2023
Nelson. Liu et al · 2023
Cited alongside, same era.
“Landmark Attention: Random-Access Infinite Context Length for Transformers”, 2023
Amirkeivan Mohtashami and Martin Jaggi · 2023
Cited alongside, same era.
“GPT 4.”, 2023
OpenAI · 2023
Cited alongside, same era.
“YaRN: Efficient Context Window Extension of Large Language Models”, 2023
Bowen Peng, Jeffrey Quesnelle, Honglu Fan and Enrico Shippole · 2023
Cited alongside, same era.
“ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding”, 2023
Uri Shaham et al · 2023
Cited alongside, same era.
Marah Abdin et al · 2024
Closest in time.
“LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding”, 2024
Yushi Bai et al · 2024
Closest in time.
“Graph of Thoughts: Solving Elaborate Problems with Large Language Models”
Maciej Besta et al · 2024
Closest in time.
“LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models”, 2024
Yukang Chen et al · 2024
Closest in time.
“The Llama 3 Herd of Models”, 2024
Abhimanyu Dubey et al · 2024
Closest in time.
“In-context Autoencoder for Context Compression in a Large Language Model”, 2024
Tao Ge et al · 2024
Closest in time.
“RULER: What’s the Real Context Size of Your Long-Context Language Models?”, 2024
Cheng-Ping Hsieh et al · 2024
Closest in time.
“Learning to Compress Prompts with Gist Tokens”, 2024
Jesse Mu, Xiang Li and Noah Goodman · 2024
Closest in time.
“Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention”, 2024
Tsendsuren Munkhdalai, Manaal Faruqui and Siddharth Gopal · 2024
Closest in time.
“MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries”, 2024
Yixuan Tang and Yi Yang · 2024
Closest in time.
“Performance and Tuning”, 2024
vLLM · 2024
Closest in time.
“Qwen2 Technical Report”, 2024
An Yang et al · 2024
Closest in time.