Built on

Evaluating Large Language Models Trained on Code
Original
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al · 2021
Earlier work this paper cites.
LoRA: Low-Rank Adaptation of Large Language Models. In ICLR 2022 . arXiv
Original
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021 · 2022
Earlier work this paper cites.
Training Language Models to Follow Instructions with Human Feedback
Original
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al · 2022
Earlier work this paper cites.
Ignore Previous Prompt: Attack Techniques For Language Models. In NeurIPS 2022 Workshop on Machine Learning Safety . arXiv
Original
Fábio Perez and Ian Ribeiro. 2022 · 2022
Earlier work this paper cites.
Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks. In EMNLP 2022 . arXiv
Original
Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al · 2022
Earlier work this paper cites.
Finetuned Language Models Are Zero-Shot Learners. In ICLR 2022 . arXiv
Original
Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. 2022 · 2022
Earlier work this paper cites.
Synthetic Python Problems(SPP) Dataset
2023 · 2023
Earlier work this paper cites.
Free Dolly: Introducing the World’s First Truly Open Instruction-Tuned LLM
Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. 2023 · 2023
Earlier work this paper cites.
Enhancing Chat Language Models by Scaling High-quality Instructional Conversations. In EMNLP 2023 . arXiv
Original
Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Zhi Zheng, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. 2023 · 2023
Earlier work this paper cites.
Not What You’ve Signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. In CCS 2023 Workshop on Artificial Intelligence and Security (AISec 2023) . arXiv
Original
Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. 2023 · 2023
Earlier work this paper cites.
DeBERTaV3: Improving DeBERTa Using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing. In ICLR 2023 . arXiv
Original
Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023 · 2023
Earlier work this paper cites.
Challenges and Applications of Large Language Models
Original
Jean Kaddour, Joshua Harris, Maximilian Mozes, Herbie Bradley, Roberta Raileanu, and Robert McHardy. 2023 · 2023
Earlier work this paper cites.
Text-Davinci-003
OpenAI. 2023 · 2023
Earlier work this paper cites.

Similar

Fine-Tuned DeBERTa-v3-base for Prompt Injection Detection
ProtectAI.com. 2023 · 2023
Cited alongside, same era.
Fine-Tuned DeBERTa-v3 for Prompt Injection Detection
ProtectAI.com. 2023 · 2023
Cited alongside, same era.
Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs through a Global Scale Prompt Hacking Competition. In EMNLP 2023 . arXiv
Original
Sander Schulhoff, Jeremy Pinto, Anaum Khan, Louis-François Bouchard, Chenglei Si, Svetlina Anati, Valen Tagliabue, Anson Liu Kost, Christopher Carnahan, and Jordan Boyd-Graber. 2024 · 2023
Cited alongside, same era.
Stanford Alpaca: An Instruction-following LLaMA Model
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori Hashimoto. 2023 · 2023
Cited alongside, same era.
LLaMA: Open and Efficient Foundation Language Models
Original
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al · 2023

Then

Omni-Moderation-Latest
OpenAI. 2024 · 2024
Later among the works it cites.
GPT-4 Technical Report
Original
OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, et al · 2024
Later among the works it cites.
Jatmo: Prompt Injection Defense by Task-Specific Finetuning. In ESORICS 2024 . arXiv
Original
Julien Piet, Maha Alrashed, Chawin Sitawarin, Sizhe Chen, Zeming Wei, Elizabeth Sun, Basel Alomair, and David Wagner. 2024 · 2024
Later among the works it cites.
Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks. In LREC-COLING 2024 . arXiv
Original
Abhinav Rao, Sachin Vashistha, Atharva Naik, Somak Aditya, and Monojit Choudhury. 2024 · 2024
Later among the works it cites.
"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models. In CCS 2024 . arXiv

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2025

PromptShield: Deployable Detection for Prompt Injection Attacks

Jacob, Dennis, Alzahrani, Hend, Hu, Zhanhao et al.

Understand

Application designers have moved to integrate large language models (LLMs) into their products.

However, many LLM-integrated applications are vulnerable to prompt injections.
While attempts have been made to address this problem by building prompt injection detectors, many are not yet suitable for practical deployment.
To support research in this area, we introduce PromptShield, a benchmark for training and evaluating deployable prompt injection detectors.

Built on

Evaluating Large Language Models Trained on Code
Original
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al · 2021
Earlier work this paper cites.
LoRA: Low-Rank Adaptation of Large Language Models. In ICLR 2022 . arXiv
Original
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021 · 2022
Earlier work this paper cites.
Training Language Models to Follow Instructions with Human Feedback
Original
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al · 2022
Earlier work this paper cites.
Ignore Previous Prompt: Attack Techniques For Language Models. In NeurIPS 2022 Workshop on Machine Learning Safety . arXiv
Original
Fábio Perez and Ian Ribeiro. 2022 · 2022
Earlier work this paper cites.
Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks. In EMNLP 2022 . arXiv
Original
Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al · 2022
Earlier work this paper cites.
Finetuned Language Models Are Zero-Shot Learners. In ICLR 2022 . arXiv
Original
Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. 2022 · 2022
Earlier work this paper cites.
Synthetic Python Problems(SPP) Dataset
2023 · 2023
Earlier work this paper cites.
Free Dolly: Introducing the World’s First Truly Open Instruction-Tuned LLM
Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. 2023 · 2023
Earlier work this paper cites.
Enhancing Chat Language Models by Scaling High-quality Instructional Conversations. In EMNLP 2023 . arXiv
Original
Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Zhi Zheng, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. 2023 · 2023
Earlier work this paper cites.
Not What You’ve Signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. In CCS 2023 Workshop on Artificial Intelligence and Security (AISec 2023) . arXiv
Original
Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. 2023 · 2023
Earlier work this paper cites.
DeBERTaV3: Improving DeBERTa Using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing. In ICLR 2023 . arXiv
Original
Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023 · 2023
Earlier work this paper cites.
Challenges and Applications of Large Language Models
Original
Jean Kaddour, Joshua Harris, Maximilian Mozes, Herbie Bradley, Roberta Raileanu, and Robert McHardy. 2023 · 2023
Earlier work this paper cites.
Text-Davinci-003
OpenAI. 2023 · 2023
Earlier work this paper cites.

Similar

Fine-Tuned DeBERTa-v3-base for Prompt Injection Detection
ProtectAI.com. 2023 · 2023
Cited alongside, same era.
Fine-Tuned DeBERTa-v3 for Prompt Injection Detection
ProtectAI.com. 2023 · 2023
Cited alongside, same era.
Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs through a Global Scale Prompt Hacking Competition. In EMNLP 2023 . arXiv
Original
Sander Schulhoff, Jeremy Pinto, Anaum Khan, Louis-François Bouchard, Chenglei Si, Svetlina Anati, Valen Tagliabue, Anson Liu Kost, Christopher Carnahan, and Jordan Boyd-Graber. 2024 · 2023
Cited alongside, same era.
Stanford Alpaca: An Instruction-following LLaMA Model
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori Hashimoto. 2023 · 2023
Cited alongside, same era.
LLaMA: Open and Efficient Foundation Language Models
Original
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al · 2023

Then

Omni-Moderation-Latest
OpenAI. 2024 · 2024
Later among the works it cites.
GPT-4 Technical Report
Original
OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, et al · 2024
Later among the works it cites.
Jatmo: Prompt Injection Defense by Task-Specific Finetuning. In ESORICS 2024 . arXiv
Original
Julien Piet, Maha Alrashed, Chawin Sitawarin, Sizhe Chen, Zeming Wei, Elizabeth Sun, Basel Alomair, and David Wagner. 2024 · 2024
Later among the works it cites.
Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks. In LREC-COLING 2024 . arXiv
Original
Abhinav Rao, Sachin Vashistha, Atharva Naik, Somak Aditya, and Monojit Choudhury. 2024 · 2024
Later among the works it cites.
"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models. In CCS 2024 . arXiv

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Cited alongside, same era.

Self-Instruct: Aligning Language Models with Self-Generated Instructions. In ACL 2023 . arXiv

Original

Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023 · 2023

Cited alongside, same era.

Instruction-Following Evaluation for Large Language Models

Original

Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. 2023 · 2023

Cited alongside, same era.

Universal and Transferable Adversarial Attacks on Aligned Language Models

Original

Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. 2023 · 2023

Cited alongside, same era.

Fmops/Distilbert-Prompt-Injection

Blueteam AI. 2024 · 2024

Cited alongside, same era.

The Llama 3 Herd of Models

Original

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al · 2024

Cited alongside, same era.

Attention Tracker: Detecting Prompt Injection Attacks in LLMs

Original

Kuo-Han Hung, Ching-Yun Ko, Ambrish Rawat, I.-Hsin Chung, Winston H. Hsu, and Pin-Yu Chen. 2024 · 2024

Cited alongside, same era.

InjecGuard: Benchmarking and Mitigating Over-defense in Prompt Injection Guardrail Models

Original

Hao Li and Xiaogeng Liu. 2024 · 2024

Cited alongside, same era.

Original

Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. 2024 · 2024