LLM In-Context Recall is Prompt Dependent

A framework for few-shot language model evaluation

Leo Gao, Jonathan Tow, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Kyle McDonell, Niklas Muennighoff, Jason Phang, Laria Reynolds, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. 2021 · 2021

Earlier work this paper cites.

Open LLM Leaderboard

Edward Beeching, Clémentine Fourrier, Nathan Habib, Sheon Han, Nathan Lambert, Nazneen Rajani, Omar Sanseviero, Lewis Tunstall, and Thomas Wolf. 2023 · 2023

Earlier work this paper cites.

Mistral 7B

Original

Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023 · 2023

Earlier work this paper cites.

Pressure Testing GPT-4-128K With Long Context Recall

Greg Kamradt. 2023 · 2023

Earlier work this paper cites.

Lost in the Middle: How Language Models Use Long Contexts

Original

Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2023 · 2023

Earlier work this paper cites.

Instruction Tuning with GPT-4

Original

Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. 2023 · 2023

Cited alongside, same era.

JudgeLM: Fine-tuned Large Language Models are Scalable Judges

Original

Lianghui Zhu, Xinggang Wang, and Xinlong Wang. 2023 · 2023

Cited alongside, same era.

Mixtral of Experts

Original

Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, Teven Le Scao, Théophile Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2024 · 2024

Cited alongside, same era.

String Evaluation Guide

LangChain. [n. d.] · 2024

Cited alongside, same era.

Multi Needle in a Haystack

LangChain. 2024 · 2024

Closest in time.

Mistral-7B-Instruct-v0.2

Mistral AI Team. 2023 · 2024

Closest in time.

New Embedding Models and API Updates

OpenAI. 2024 · 2024

Closest in time.

Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

Original

Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, and Timothy Lillicrap et al. 2024 · 2024

Closest in time.