Built on

Deep reinforcement learning from human preferences
Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei · 2017
Earlier work this paper cites.
Gradient-based adversarial attacks against text transformers
Original
Chuan Guo, Alexandre Sablayrolles, Hervé Jégou, and Douwe Kiela · 2021
Earlier work this paper cites.
Probing toxic content in large pre-trained language models
Nedjma Ousidhoum, Xinran Zhao, Tianqing Fang, Yangqiu Song, and Dit-Yan Yeung · 2021
Earlier work this paper cites.
Finetuned language models are zero-shot learners
Original
Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le · 2021
Earlier work this paper cites.
Training a helpful and harmless assistant with reinforcement learning from human feedback
Original
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al · 2022
Earlier work this paper cites.
Training language models to follow instructions with human feedback
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al · 2022
Earlier work this paper cites.
Jailbreaking chatgpt on release day
Zvi · 2022
Earlier work this paper cites.
Gpt-4 technical report
Original
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al · 2023
Earlier work this paper cites.

Similar

Detecting language model attacks with perplexity
Original
Gabriel Alon and Michael Kamfonas · 2023
Cited alongside, same era.
Llm self defense: By self examination, llms know they are being tricked
Original
Alec Helbling, Mansi Phute, Matthew Hull, and Duen Horng Chau · 2023
Cited alongside, same era.
Catastrophic jailbreak of open-source llms via exploiting generation
Original
Yangsibo Huang, Samyak Gupta, Mengzhou Xia, Kai Li, and Danqi Chen · 2023
Cited alongside, same era.
Baseline defenses for adversarial attacks against aligned language models
Original
Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping-yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, and Tom Goldstein · 2023
Cited alongside, same era.
Mistral 7b

Then

Jailbreak and guard aligned language models with only few in-context demonstrations
Original
Zeming Wei, Yifei Wang, and Yisen Wang · 2023
Later among the works it cites.
Defending chatgpt against jailbreak attack via self-reminders
Yueqi Xie, Jingwei Yi, Jiawei Shao, Justin Curl, Lingjuan Lyu, Qifeng Chen, Xing Xie, and Fangzhao Wu · 2023
Later among the works it cites.
Gpt-4 is too smart to be safe: Stealthy chat with llms via cipher
Original
Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Pinjia He, Shuming Shi, and Zhaopeng Tu · 2023
Later among the works it cites.
Universal and transferable adversarial attacks on aligned language models
Original
Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrikson · 2023
Later among the works it cites.
Foundational challenges in assuring alignment and safety of large language models

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2024

DROJ: A Prompt-Driven Attack against Large Language Models

Hu, Leyang, Wang, Boran

Understand

Large Language Models (LLMs) have demonstrated exceptional capabilities across various natural language processing tasks.

Due to their training on internet-sourced datasets, LLMs can sometimes generate objectionable content, necessitating extensive alignment with human feedback to avoid such outputs.
Despite massive alignment efforts, LLMs remain susceptible to adversarial jailbreak attacks, which usually are manipulated prompts designed to circumvent safety mechanisms and elicit harmful responses.
Here, we introduce a novel approach, Directed Rrepresentation Optimization Jailbreak (DROJ), which optimizes jailbreak prompts at the embedding level to shift the hidden representations of harmful queries towards directions that are more likely to elicit affirmative responses from the model.

Built on

Deep reinforcement learning from human preferences
Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei · 2017
Earlier work this paper cites.
Gradient-based adversarial attacks against text transformers
Original
Chuan Guo, Alexandre Sablayrolles, Hervé Jégou, and Douwe Kiela · 2021
Earlier work this paper cites.
Probing toxic content in large pre-trained language models
Nedjma Ousidhoum, Xinran Zhao, Tianqing Fang, Yangqiu Song, and Dit-Yan Yeung · 2021
Earlier work this paper cites.
Finetuned language models are zero-shot learners
Original
Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le · 2021
Earlier work this paper cites.
Training a helpful and harmless assistant with reinforcement learning from human feedback
Original
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al · 2022
Earlier work this paper cites.
Training language models to follow instructions with human feedback
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al · 2022
Earlier work this paper cites.
Jailbreaking chatgpt on release day
Zvi · 2022
Earlier work this paper cites.
Gpt-4 technical report
Original
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al · 2023
Earlier work this paper cites.

Similar

Detecting language model attacks with perplexity
Original
Gabriel Alon and Michael Kamfonas · 2023
Cited alongside, same era.
Llm self defense: By self examination, llms know they are being tricked
Original
Alec Helbling, Mansi Phute, Matthew Hull, and Duen Horng Chau · 2023
Cited alongside, same era.
Catastrophic jailbreak of open-source llms via exploiting generation
Original
Yangsibo Huang, Samyak Gupta, Mengzhou Xia, Kai Li, and Danqi Chen · 2023
Cited alongside, same era.
Baseline defenses for adversarial attacks against aligned language models
Original
Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping-yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, and Tom Goldstein · 2023
Cited alongside, same era.
Mistral 7b

Then

Jailbreak and guard aligned language models with only few in-context demonstrations
Original
Zeming Wei, Yifei Wang, and Yisen Wang · 2023
Later among the works it cites.
Defending chatgpt against jailbreak attack via self-reminders
Yueqi Xie, Jingwei Yi, Jiawei Shao, Justin Curl, Lingjuan Lyu, Qifeng Chen, Xing Xie, and Fangzhao Wu · 2023
Later among the works it cites.
Gpt-4 is too smart to be safe: Stealthy chat with llms via cipher
Original
Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Pinjia He, Shuming Shi, and Zhaopeng Tu · 2023
Later among the works it cites.
Universal and transferable adversarial attacks on aligned language models
Original
Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrikson · 2023
Later among the works it cites.
Foundational challenges in assuring alignment and safety of large language models

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Original

Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al · 2023

Cited alongside, same era.

Open sesame! universal black box jailbreaking of large language models

Original

Raz Lapid, Ron Langberg, and Moshe Sipper · 2023

Cited alongside, same era.

Autodan: Generating stealthy jailbreak prompts on aligned large language models

Original

Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao · 2023

Cited alongside, same era.

Scalable and transferable black-box jailbreaks for language models via persona modulation

Original

Rusheb Shah, Soroush Pour, Arush Tagade, Stephen Casper, Javier Rando, et al · 2023

Cited alongside, same era.

Original

Usman Anwar, Abulhair Saparov, Javier Rando, Daniel Paleka, Miles Turpin, Peter Hase, Ekdeep Singh Lubana, Erik Jenner, Stephen Casper, Oliver Sourbut, et al · 2024

Closest in time.

From noise to clarity: Unraveling the adversarial suffix of large language model attacks via translation of text embeddings

Original

Hao Wang, Hao Li, Minlie Huang, and Lei Sha · 2024

Closest in time.

Jailbroken: How does llm safety training fail?

Alexander Wei, Nika Haghtalab, and Jacob Steinhardt · 2024

Closest in time.

Prompt-driven llm safeguarding via directed representation optimization

Original

Chujie Zheng, Fan Yin, Hao Zhou, Fandong Meng, Jie Zhou, Kai-Wei Chang, Minlie Huang, and Nanyun Peng · 2024

Closest in time.