Fetching the paper…
Reading the bibliography…
Previous work finds that recent long-context language models fail to make equal use of information in the middle of their inputs, preferring pieces of information located at the tail ends which creates an undue bias in situations where we would like models to be equally capable of using different parts of the input.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2019 · 1910
Earlier work this paper cites.
Leveraging passage retrieval with generative models for open domain question answering
Gautier Izacard and Edouard Grave. 2020 · 2007
Earlier work this paper cites.
Teaching machines to read and comprehend
Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015 · 2015
Earlier work this paper cites.
HotpotQA: A dataset for diverse, explainable multi-hop question answering
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018 · 2018
Earlier work this paper cites.
Natural questions: A benchmark for question answering research
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019 · 2019
Earlier work this paper cites.
Improving multi-hop question answering over knowledge graphs using knowledge base embeddings
Apoorv Saxena, Aditay Tripathi, and Partha Talukdar. 2020 · 2020
Earlier work this paper cites.
Is multihop QA in DiRe condition? measuring and reducing disconnected reasoning
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2020 · 2020
Earlier work this paper cites.
Robustifying multi-hop qa through pseudo-evidentiality training
Kyungjae Lee, Seung won Hwang, Sang eun Han, and Dohyeon Lee. 2021 · 2021
Earlier work this paper cites.
Flashattention: Fast and memory-efficient exact attention with io-awareness
Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. 2022 · 2022
Earlier work this paper cites.
Counterfactual multihop qa: A cause-effect approach for reducing disconnected reasoning
Wangzhen Guo, Qinkang Gong, and Hanjiang Lai. 2022 · 2022
Earlier work this paper cites.
Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2022 · 2022
Earlier work this paper cites.
Train short, test long: Attention with linear biases enables input length extrapolation
Ofir Press, Noah Smith, and Mike Lewis. 2022 · 2022
Cited alongside, same era.
Longlora: Efficient fine-tuning of long-context large language models
Yukang Chen, Shengju Qian, Haotian Tang, Xin Lai, Zhijian Liu, Song Han, and Jiaya Jia. 2023 · 2023
Cited alongside, same era.
Flashattention-2: Faster attention with better parallelism and work partitioning
Tri Dao. 2023 · 2023
Cited alongside, same era.
FactKB: Generalizable factuality evaluation using language models enhanced with factual knowledge
Shangbin Feng, Vidhisha Balachandran, Yuyang Bai, and Yulia Tsvetkov. 2023 · 2023
Cited alongside, same era.
Large language models struggle to learn long-tail knowledge
Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. 2023 · 2023
Cited alongside, same era.
Least-to-most prompting enables complex reasoning in large language models
Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, and Ed Chi. 2023 · 2023
Later among the works it cites.
Make your llm fully utilize the context
Shengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng, and Jian-Guang Lou. 2024 · 2024
Closest in time.
Sure: Improving open-domain question answering of LLMs via summarized retrieval
Jaehyung Kim, Jaehyun Nam, Sangwoo Mo, Jongjin Park, Sang-Woo Lee, Minjoon Seo, Jung-Woo Ha, and Jinwoo Shin. 2024 · 2024
Closest in time.
Mosh Levy, Alon Jacoby, and Yoav Goldberg. 2024 · 2024
Closest in time.
Lost in the middle: How language models use long contexts
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Attention sorting combats recency bias in long context language models
Alexander Peysakhovich and Adam Lerer. 2023 · 2023
Cited alongside, same era.
Measuring and modifying factual knowledge in large language models
Pouya Pezeshkpour. 2023 · 2023
Cited alongside, same era.
Found in the middle: Permutation self-consistency improves listwise ranking in large language models
Raphael Tang, Xinyu Zhang, Xueguang Ma, Jimmy Lin, and Ferhan Ture. 2023 · 2023
Cited alongside, same era.
Llama 2: Open foundation and fine-tuned chat models
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023 · 2023
Cited alongside, same era.
Survey on factuality in large language models: Knowledge, retrieval and domain-specificity
Cunxiang Wang, Xiaoze Liu, Yuanhao Yue, Xiangru Tang, Tianhang Zhang, Cheng Jiayang, Yunzhi Yao, Wenyang Gao, Xuming Hu, Zehan Qi, Yidong Wang, Linyi Yang, Jindong Wang, Xing Xie, Zheng Zhang, and Yue Zhang. 2023 · 2023
Cited alongside, same era.
Emerging challenges in personalized medicine: Assessing demographic effects on biomedical question answering systems
Sagi Shaier, Kevin Bennett, Lawrence Hunter, and Katharina Kann. 2023a
Cited in the paper.
Comparing template-based and template-free language model probing
Sagi Shaier, Kevin Bennett, Lawrence Hunter, and Katharina von der Wense. 2024a
Cited in the paper.
Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024 · 2024
Closest in time.
Vaibhav Mavi, Anubhav Jangra, and Adam Jatowt. 2024 · 2024
Closest in time.
It is not about what you say, it is about how you say it: A surprisingly simple approach for improving reading comprehension
Sagi Shaier, Lawrence Hunter, and Katharina Wense. 2024c · 2024
Closest in time.
Adaptive question answering: Enhancing language model proficiency for addressing knowledge conflicts with source citations
Sagi Shaier, Ari Kobren, and Philip V. Ogren. 2024d · 2024
Closest in time.
Xin Su, Tiep Le, Steven Bethard, and Phillip Howard. 2024 · 2024
Closest in time.
Factuality of large language models: A survey
Yuxia Wang, Minghan Wang, Muhammad Arslan Manzoor, Fei Liu, Georgi Nenkov Georgiev, Rocktim Jyoti Das, and Preslav Nakov. 2024 · 2024
Closest in time.