Tool Preferences in Agentic LLMs are Unreliable

Evaluating the susceptibility of pre-trained language models via handcrafted adversarial examples

Original

Hezekiah J Branch, Jonathan Rodriguez Cefalu, Jeremy McHugh, Leyla Hujer, Aditya Bahl, Daniel del Castillo Iglesias, Ron Heichman, and Ramesh Darwishi. 2022 · 2022

Earlier work this paper cites.

Langchain: Building applications with llms through composability

LangChain. 2022 · 2022

Earlier work this paper cites.

LlamaIndex

Jerry Liu. 2022 · 2022

Earlier work this paper cites.

Talm: Tool augmented language models

Original

Aaron Parisi, Yao Zhao, and Noah Fiedel. 2022 · 2022

Earlier work this paper cites.

Ignore previous prompt: Attack techniques for language models

Original

Fábio Perez and Ian Ribeiro. 2022 · 2022

Earlier work this paper cites.

Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection

Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. 2023 · 2023

Earlier work this paper cites.

Augmented language models: a survey

Original

Grégoire Mialon, Roberto Dessì, Maria Lomeli, Christoforos Nalmpantis, Ram Pasunuru, Roberta Raileanu, Baptiste Rozière, Timo Schick, Jane Dwivedi-Yu, Asli Celikyilmaz, and 1 others. 2023 · 2023

Earlier work this paper cites.

Function calling and other api updates

OpenAI. 2023 · 2023

Earlier work this paper cites.

Toolllm: Facilitating large language models to master 16000+ real-world apis

Original

Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, and 1 others. 2023 · 2023

Earlier work this paper cites.

Toolformer: Language models can teach themselves to use tools

Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023 · 2023

Earlier work this paper cites.

Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face

Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. 2023 · 2023

Cited alongside, same era.

Restgpt: Connecting large language models with real-world restful apis

Original

Yifan Song, Weimin Xiong, Dawei Zhu, Wenhao Wu, Han Qian, Mingbo Song, Hailiang Huang, Cheng Li, Ke Wang, Rong Yao, and 1 others. 2023 · 2023

Cited alongside, same era.

Introducing the model context protocol

Anthropic. 2024 · 2024

Cited alongside, same era.

Bitagent-8b

BitAgent. 2024 · 2024

Cited alongside, same era.

The llama 3 herd of models

Original

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024 · 2024

Cited alongside, same era.

Gorilla: Large language model connected with massive apis

Shishir G Patil, Tianjun Zhang, Xin Wang, and Joseph E Gonzalez. 2024 · 2024

Later among the works it cites.

Tool learning with foundation models

Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding, Ganqu Cui, Zheni Zeng, Xuanhe Zhou, Yufei Huang, Chaojun Xiao, and 1 others. 2024 · 2024

Later among the works it cites.

Qwen2.5: A party of foundation models

Qwen Team. 2024 · 2024

Later among the works it cites.

watt-tool-8b

watt ai. 2024 · 2024

Later among the works it cites.

Berkeley function calling leaderboard

Fanjia Yan, Huanzhi Mao, Charlie Cheng-Jie Ji, Tianjun Zhang, Shishir G. Patil, Ion Stoica, and Joseph E. Gonzalez. 2024 · 2024

Later among the works it cites.

Injecagent: Benchmarking indirect prompt injections in tool-integrated large language model agents

Original

Qiusi Zhan, Zhixiang Liang, Zifan Ying, and Daniel Kang. 2024 · 2024