2023

Image Hijacks: Adversarial Images can Control Generative Models at Runtime

Bailey, Luke, Ong, Euan, Russell, Stuart et al.

Understand

Are foundation models secure against malicious actors? In this work, we focus on the image input to a vision-language model (VLM).

We discover image hijacks, adversarial images that control the behaviour of VLMs at inference time, and introduce the general Behaviour Matching algorithm for training image hijacks.
From this, we derive the Prompt Matching method, allowing us to train hijacks matching the behaviour of an arbitrary user-defined text prompt (e.g.
'the Eiffel Tower is now located in Rome') using a generic, off-the-shelf dataset unrelated to our choice of prompt.

Built on

On Evaluating Adversarial Robustness, feb 2019
Original
Carlini, N., Athalye, A., Papernot, N., Brendel, W., Rauber, J., Tsipras, D., Goodfellow, I., Ma̧dry, A., Kurakin, A., Brain, G., Evaluating, O., and Robustness, A · 1902
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L · 2009
Earlier work this paper cites.
Pillow (pil fork) documentation, 2015
Clark, A · 2015
Earlier work this paper cites.
Distilling the knowledge in a neural network, 2015
Hinton, G., Vinyals, O., and Dean, J · 2015
Earlier work this paper cites.
Countering adversarial images using input transformations
Original
Guo, C., Rana, M., Cisse, M., and Van Der Maaten, L · 2017
Earlier work this paper cites.
Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples
Athalye, A., Carlini, N., and Wagner, D · 2018
Earlier work this paper cites.
Certified adversarial robustness via randomized smoothing
Cohen, J., Rosenfeld, E., and Kolter, Z · 2019
Earlier work this paper cites.
Small input noise is enough to defend against query-based black-box attacks, 2020
Byun, J., Go, H., and Kim, C · 2020
Earlier work this paper cites.
Robustbench: a standardized adversarial robustness benchmark
Original
Croce, F., Andriushchenko, M., Sehwag, V., Debenedetti, E., Flammarion, N., Chiang, M., Mittal, P., and Hein, M · 2020
Earlier work this paper cites.
Uncovering the limits of adversarial training against norm-bounded adversarial examples
Original
Gowal, S., Qin, C., Uesato, J., Mann, T., and Kohli, P · 2020
Earlier work this paper cites.
Bert-attack: Adversarial attack against bert using bert
Original
Li, L., Ma, R., Guo, Q., Xue, X., and Qiu, X · 2020
Earlier work this paper cites.
Random noise defense against query-based black-box attacks
Qin, Z., Fan, Y., Zha, H., and Wu, B · 2021
Earlier work this paper cites.
Learning transferable visual models from natural language supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al · 2021
Earlier work this paper cites.

Similar

(certified!!) adversarial robustness for free!
Original
Carlini, N., Tramer, F., Dvijotham, K. D., Rice, L., Sun, M., and Kolter, J. Z · 2022
Cited alongside, same era.
LangChain, October 2022
Chase, H · 2022
Cited alongside, same era.
Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning, 2022
Liang, W., Zhang, Y., Kwon, Y., Yeung, S., and Zou, J · 2022
Cited alongside, same era.
Training language models to follow instructions with human feedback, 2022
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., and Lowe, R · 2022
Cited alongside, same era.
Ignore Previous Prompt: Attack Techniques For Language Models, nov 2022
Original
Perez, F. and Ribeiro, I · 2022
Cited alongside, same era.
(ab) using images and sounds for indirect instruction injection in multi-modal llms
Original
Bagdasaryan, E., Hsieh, T.-Y., Nassi, B., and Shmatikov, V · 2023
Cited alongside, same era.
Are aligned neural networks adversarially aligned?
Original
Carlini, N., Nasr, M., Choquette-Choo, C. A., Jagielski, M., Gao, I., Awadalla, A., Koh, P. W., Ippolito, D., Lee, K., Tramer, F., et al · 2023
Cited alongside, same era.
Instructblip: Towards general-purpose vision-language models with instruction tuning
Original
Dai, W., Li, J., Li, D., Meng Huat Tiong, A., Zhao, J., Wang, W., Li, B., Fung, P., and Hoi, S · 2023
Cited alongside, same era.
How robust is google’s bard to adversarial image attacks?
Original
Dong, Y., Chen, H., Chen, J., Fang, Z., Yang, X., Zhang, Y., Tian, Y., Su, H., and Zhu, J · 2023
Cited alongside, same era.
Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection, feb 2023
Original
Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., and Fritz, M · 2023
Cited alongside, same era.
Automatically auditing large language models via discrete optimization
Original
Jones, E., Dragan, A., Raghunathan, A., and Steinhardt, J · 2023
Cited alongside, same era.
Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models
Original
Li, J., Li, D., Savarese, S., and Hoi, S · 2023
Cited alongside, same era.

Then

Augmented language models: a survey
Original
Mialon, G., Dessi, R., Lomeli, M., Nalmpantis, C., Pasunuru, R., Raileanu, R., Rozière, B., Schick, T., Dwivedi-Yu, J., Celikyilmaz, A., et al · 2023
Closest in time.
Gpt-4 technical report, 2023
OpenAI · 2023
Closest in time.
From Prompt Injections to SQL Injection Attacks: How Protected is Your LLM-Integrated Web Application?
Original
Pedro, R., Castro, D., Carreira, P., and Santos, N · 2023
Closest in time.
Google I/O 2023: Making AI more helpful for everyone
Pichai, S · 2023
Closest in time.
On the adversarial robustness of multi-modal foundation models
Original
Schlarmann, C. and Hein, M · 2023
Closest in time.
Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models
Original
Shayegani, E., Dong, Y., and Abu-Ghazaleh, N · 2023
Closest in time.
Stanford alpaca: An instruction-following llama model
Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., and Hashimoto, T. B · 2023
Closest in time.
Llama 2: Open foundation and fine-tuned chat models
Original
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al · 2023
Closest in time.
Better diffusion models further improve adversarial training
Original
Wang, Z., Pang, T., Du, C., Lin, M., Liu, W., and Yan, S · 2023
Closest in time.
Jailbroken: How does llm safety training fail?
Original
Wei, A., Haghtalab, N., and Steinhardt, J · 2023
Closest in time.
On Evaluating Adversarial Robustness of Large Vision-Language Models, 2023
Original
Zhao, Y., Pang, T., Du, C., Yang, X., Li, C., Cheung, N.-M., and Lin, M · 2023
Closest in time.
Universal and transferable adversarial attacks on aligned language models, 2023
Zou, A., Wang, Z., Kolter, J. Z., and Fredrikson, M · 2023
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…