Fetching the paper…
Reading the bibliography…
Retrieval-augmented language models (RALMs) have recently shown great potential in mitigating the limitations of implicit knowledge in LLMs, such as untimely updating of the latest expertise and unreliable retention of long-tail knowledge.
REALM: Retrieval-Augmented Language Model Pre-Training
Guu, K.; Lee, K.; Tung, Z.; Pasupat, P.; and Chang, M.-W. 2020a · 2002
Earlier work this paper cites.
REALM: Retrieval-augmented language model pre-training
Guu, K.; Lee, K.; Tung, Z.; Pasupat, P.; and Chang, M.-W. 2020b · 2002
Earlier work this paper cites.
Semantic Parsing on Freebase from Question-Answer Pairs
Berant, J.; Chou, A.; Frostig, R.; and Liang, P. 2013 · 2013
Earlier work this paper cites.
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
Joshi, M.; Choi, E.; Weld, D.; and Zettlemoyer, L. 2017 · 2017
Earlier work this paper cites.
Zero-Shot Relation Extraction via Reading Comprehension
Levy, O.; Seo, M.; Choi, E.; and Zettlemoyer, L. 2017 · 2017
Earlier work this paper cites.
Know What You Don’t Know: Unanswerable Questions for SQuAD
Rajpurkar, P.; Jia, R.; and Liang, P. 2018 · 2018
Earlier work this paper cites.
CoQA: A Conversational Question Answering Challenge
Reddy, S.; Chen, D.; and Manning, C. D. 2018 · 2018
Earlier work this paper cites.
QuaRel: A Dataset and Models for Answering Questions about Qualitative Relationships
Tafjord, O.; Clark, P.; Gardner, M.; tau Yih, W.; and Sabharwal, A. 2018 · 2018
Earlier work this paper cites.
WikiQA: A Challenge Dataset for Open-Domain Question Answering
Yang, Y.; Yih, W.-t.; and Meek, C. 2015 · 2018
Earlier work this paper cites.
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Yang, Z.; Qi, P.; Zhang, S.; Bengio, Y.; Cohen, W. W.; Salakhutdinov, R.; and Manning, C. D. 2018 · 2018
Earlier work this paper cites.
DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs
Dua, D.; Wang, Y.; Dasigi, P.; Stanovsky, G.; Singh, S.; and Gardner, M. 2019 · 2019
Earlier work this paper cites.
FreebaseQA: A New Factoid QA Data Set Matching Trivia-Style Question-Answer Pairs with Freebase
Jiang, K.; Wu, D.; and Jiang, H. 2019 · 2019
Earlier work this paper cites.
Natural Questions: A Benchmark for Question Answering Research
Kwiatkowski, T.; Palomaki, J.; Redfield, O.; Collins, M.; Parikh, A.; Alberti, C.; Epstein, D.; Polosukhin, I.; Devlin, J.; Lee, K.; Toutanova, K.; Jones, L.; Kelcey, M.; Chang, M.-W.; Dai, A. M.; Uszkoreit, J.; Le, Q.; and Petrov, S. 2019 · 2019
Earlier work this paper cites.
CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge
Talmor, A.; Herzig, J.; Lourie, N.; and Berant, J. 2019 · 2019
Cited alongside, same era.
Extracting Training Data from Large Language Models
Carlini, N.; Tramèr, F.; Wallace, E.; Jagielski, M.; Herbert-Voss, A.; Lee, K.; Roberts, A.; Brown, T. B.; Song, D. X.; Erlingsson, Ú.; Oprea, A.; and Raffel, C. 2020 · 2020
Cited alongside, same era.
Generalization through Memorization: Nearest Neighbor Language Models
Khandelwal, U.; Levy, O.; Jurafsky, D.; Zettlemoyer, L.; and Lewis, M. 2020 · 2020
Cited alongside, same era.
KILT: a Benchmark for Knowledge Intensive Language Tasks
Petroni, F.; Piktus, A.; Fan, A.; Lewis, P.; Yazdani, M.; Cao, N. D.; Thorne, J.; Jernite, Y.; Plachouras, V.; Rocktaschel, T.; and Riedel, S. 2020 · 2020
Cited alongside, same era.
Explanations for CommonsenseQA: New Dataset and Models
Aggarwal, S.; Mandowara, D.; Agrawal, V.; Khandelwal, D.; Singla, P.; and Garg, D. 2021 · 2021
Cited alongside, same era.
PaLM 2 Technical Report
Anil, R.; Dai, A. M.; Firat, O.; Johnson, M.; Lepikhin, D.; Passos, A.; Shakeri, S.; Taropa, E.; Bailey, P.; Chen, Z.; Chu, E.; Clark, J. H.; Shafey, L. E.; Huang, Y.; Meier-Hellstern, K.; Mishra, G.; Moreira, E.; Omernick, M.; Robinson, K.; Ruder, S.; Tay, Y.; Xiao, K.; Xu, Y.; Zhang, Y.; Abrego, G. H.; Ahn, J.; Austin, J.; Barham, P.; Botha, J.; Bradbury, J.; Brahma, S.; Brooks, K.; Catasta, M.; Cheng, Y.; Cherry, C.; Choquette-Choo, C. A.; Chowdhery, A.; Crepy, C.; Dave, S.; Dehghani, M.; Dev, S.; Devlin, J.; Díaz, M.; Du, N.; Dyer, E.; Feinberg, V.; Feng, F.; Fienber, V.; Freitag, M.; Garcia, X.; Gehrmann, S.; Gonzalez, L.; Gur-Ari, G.; Hand, S.; Hashemi, H.; Hou, L.; Howland, J.; Hu, A.; Hui, J.; Hurwitz, J.; Isard, M.; Ittycheriah, A.; Jagielski, M.; Jia, W.; Kenealy, K.; Krikun, M.; Kudugunta, S.; Lan, C.; Lee, K.; Lee, B.; Li, E.; Li, M.; Li, W.; Li, Y.; Li, J.; Lim, H.; Lin, H.; Liu, Z.; Liu, F.; Maggioni, M.; Mahendru, A.; Maynez, J.; Misra, V.; Moussalem, M.; Nado, Z.; Nham, J.; Ni, E.; Nystrom, A.; Parrish, A.; Pellat, M.; Polacek, M.; Polozov, A.; Pope, R.; Qiao, S.; Reif, E.; Richter, B.; Riley, P.; Ros, A. C.; Roy, A.; Saeta, B.; Samuel, R.; Shelby, R.; Slone, A.; Smilkov, D.; So, D. R.; Sohn, D.; Tokumine, S.; Valter, D.; Vasudevan, V.; Vodrahalli, K.; Wang, X.; Wang, P.; Wang, Z.; Wang, T.; Wieting, J.; Wu, Y.; Xu, K.; Xu, Y.; Xue, L.; Yin, P.; Yu, J.; Zhang, Q.; Zheng, S.; Zheng, C.; Zhou, W.; Zhou, D.; Petrov, S.; and Wu, Y. 2023 · 2023
Later among the works it cites.
Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection
Asai, A.; Wu, Z.; Wang, Y.; Sil, A.; and Hajishirzi, H. 2023 · 2023
Later among the works it cites.
Walking Down the Memory Maze: Beyond Context Limit through Interactive Reading
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Improving language models by retrieving from trillions of tokens
Borgeaud, S.; Mensch, A.; Hoffmann, J.; Cai, T.; Rutherford, E.; Millican, K.; van den Driessche, G.; Lespiau, J.-B.; Damoc, B.; Clark, A.; de Las Casas, D.; Guy, A.; Menick, J.; Ring, R.; Hennigan, T. W.; Huang, S.; Maggiore, L.; Jones, C.; Cassirer, A.; Brock, A.; Paganini, M.; Irving, G.; Vinyals, O.; Osindero, S.; Simonyan, K.; Rae, J. W.; Elsen, E.; and Sifre, L. 2021 · 2021
Cited alongside, same era.
Training Verifiers to Solve Math Word Problems
Cobbe, K.; Kosaraju, V.; Bavarian, M.; Chen, M.; Jun, H.; Kaiser, L.; Plappert, M.; Tworek, J.; Hilton, J.; Nakano, R.; Hesse, C.; and Schulman, J. 2021 · 2021
Cited alongside, same era.
Measuring Massive Multitask Language Understanding
Hendrycks, D.; Burns, C.; Basart, S.; Zou, A.; Mazeika, M.; Song, D.; and Steinhardt, J. 2021 · 2021
Cited alongside, same era.
Are Large Pre-Trained Language Models Leaking Your Personal Information?
Huang, J.; Shao, H.; and Chang, K. C.-C. 2022 · 2022
Cited alongside, same era.
DAMO-NLP at NLPCC-2022 Task 2: Knowledge Enhanced Robust NER for Speech Entity Linking
Huang, S.; Zhai, Y.; Long, X.; Jiang, Y.; Wang, X.; Zhang, Y.; and Xie, P. 2022 · 2022
Cited alongside, same era.
Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP
Khattab, O.; Santhanam, K.; Li, X. L.; Hall, D.; Liang, P.; Potts, C.; and Zaharia, M. 2022 · 2022
Cited alongside, same era.
Self-Instruct: Aligning Language Models with Self-Generated Instructions
Wang, Y.; Kordi, Y.; Mishra, S.; Liu, A.; Smith, N. A.; Khashabi, D.; and Hajishirzi, H. 2022 · 2022
Cited alongside, same era.
Chen, H.; Pasunuru, R.; Weston, J.; and Celikyilmaz, A. 2023 · 2023
Later among the works it cites.
LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models
Jiang, H.; Wu, Q.; Lin, C.-Y.; Yang, Y.; and Qiu, L. 2023 · 2023
Later among the works it cites.
Efficient Memory Management for Large Language Model Serving with PagedAttention
Kwon, W.; Li, Z.; Zhuang, S.; Sheng, Y.; Zheng, L.; Yu, C. H.; Gonzalez, J. E.; Zhang, H.; and Stoica, I. 2023 · 2023
Later among the works it cites.
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
Rafailov, R.; Sharma, A.; Mitchell, E.; Ermon, S.; Manning, C. D.; and Finn, C. 2023 · 2023
Later among the works it cites.
LLaMA: Open and Efficient Foundation Language Models
Touvron, H.; Lavril, T.; Izacard, G.; Martinet, X.; Lachaux, M.-A.; Lacroix, T.; Rozière, B.; Goyal, N.; Hambro, E.; Azhar, F.; Rodriguez, A.; Joulin, A.; Grave, E.; and Lample, G. 2023 · 2023
Later among the works it cites.
Zephyr: Direct Distillation of LM Alignment
Tunstall, L.; Beeching, E.; Lambert, N.; Rajani, N.; Rasul, K.; Belkada, Y.; Huang, S.; von Werra, L.; Fourrier, C.; Habib, N.; Sarrazin, N.; Sanseviero, O.; Rush, A. M.; and Wolf, T. 2023 · 2023
Later among the works it cites.
RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentation
Xu, F.; Shi, W.; and Choi, E. 2023 · 2023
Later among the works it cites.
Retrieval meets Long Context Large Language Models
Xu, P.; Ping, W.; Wu, X.; McAfee, L. C.; Zhu, C.; Liu, Z.; Subramanian, S.; Bakhturina, E.; Shoeybi, M.; and Catanzaro, B. 2023 · 2023
Later among the works it cites.
Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models
Yu, W.; Zhang, H.; Pan, X.; Ma, K.; Wang, H.; and Yu, D. 2023 · 2023
Later among the works it cites.
LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression
Jiang, H.; Wu, Q.; Luo, X.; Li, D.; Lin, C.-Y.; Yang, Y.; and Qiu, L. 2024 · 2024
Closest in time.
Jin, J.; Zhu, Y.; Zhou, Y.; and Dou, Z. 2024 · 2024
Closest in time.