Fetching the paper…

Reading the bibliography…

2025

Refusal Behavior in Large Language Models: A Nonlinear Perspective

Hildebrandt, Fabian, Maier, Andreas, Krauss, Patrick et al.

Original
PDF
Post on X

Understand

Refusal behavior in large language models (LLMs) enables them to decline responding to harmful, unethical, or inappropriate prompts, ensuring alignment with ethical standards.

This paper investigates refusal behavior across six LLMs from three architectural families.
We challenge the assumption of refusal as a linear phenomenon by employing dimensionality reduction techniques, including PCA, t-SNE, and UMAP.
Our results reveal that refusal mechanisms exhibit nonlinear, multidimensional characteristics that vary by model architecture and layer.

Refusal Behavior in Large Language Models: A Nonlinear Perspective · Around

Built on

K. Pearson, “On lines and planes of closest fit to systems of points in space,” Philosophical Magazine , vol. 2, no. 11, pp. 559–572, 1901
1901
Earlier work this paper cites.
R. Blair, “The amygdala and ventromedial prefrontal cortex in morality and psychopathy,” Trends in Cognitive Sciences , vol. 11, no. 9, pp. 387–392, 2007, opinion article
2007
Earlier work this paper cites.
A. Glenn, A. Raine, and R. Schug, “The neural correlates of moral decision-making in psychopathy,” Molecular Psychiatry , vol. 14, pp. 5–6, 2009, published 19 December 2008. [Online]. Available: https://doi.org/10.1038/mp.2008.104
2008
Earlier work this paper cites.
L. van der Maaten and G. Hinton, “Visualizing data using t-SNE,” Journal of Machine Learning Research , vol. 9, pp. 2579–2605, 2008
2008
Earlier work this paper cites.
S. Kim and D. Lee, “Prefrontal cortex and impulsive decision making,” Biological Psychiatry , vol. 69, no. 12, pp. 1140–1146, 2011, epub 2010 Aug 21
2010
Earlier work this paper cites.
M. Faruqui, Y. Tsvetkov, D. Yogatama, C. Dyer, and N. Smith, “Sparse overcomplete word vector representations,” 2015. [Online]. Available: https://arxiv.org/abs/1506.02004
Original
2015
Earlier work this paper cites.
G. Goh, “Decoding the thought vector,” https://gabgoh.github.io/ThoughtVectors/, 2016
2016
Earlier work this paper cites.
S. Arora, Y. Li, Y. Liang, T. Ma, and A. Risteski, “Linear algebraic structure of word senses, with applications to polysemy,” 2018. [Online]. Available: https://arxiv.org/abs/1601.03764
Original
2018
Earlier work this paper cites.
L. McInnes, J. Healy, and J. Melville, “Umap: Uniform manifold approximation and projection for dimension reduction,” arXiv preprint arXiv:1802.03426 , 2018
Original
2018
Earlier work this paper cites.
C. Olah, N. Cammarata, L. Schubert, G. Goh, M. Petrov, and S. Carter, “Zoom in: An introduction to circuits,” Distill , 2020, https://distill.pub/2020/circuits/zoom-in
2020
Earlier work this paper cites.
A. Schilling, A. Maier, R. Gerum, C. Metzner, and P. Krauss, “Quantifying the separability of data classes in neural networks,” Neural Networks , vol. 139, pp. 278–293, July 2021, epub 2021 Apr 5
2021
Earlier work this paper cites.

Similar

N. Elhage, T. Hume, C. Olsson, N. Schiefer, T. Henighan, S. Kravec, Z. Hatfield-Dodds, R. Lasenby, D. Drain, C. Chen, R. Grosse, S. McCandlish, J. Kaplan, D. Amodei, M. Wattenberg, and C. Olah, “Toy models of superposition,” 2022. [Online]. Available: https://arxiv.org/abs/2209.10652
Original
2022
Cited alongside, same era.
N. Nanda and J. Bloom, “Transformerlens,” https://github.com/TransformerLensOrg/TransformerLens, 2022
2022
Cited alongside, same era.
T. Bricken, A. Templeton, J. Batson, B. Chen, A. Jermyn, T. Conerly, N. Turner, C. Anil, C. Denison, A. Askell, R. Lasenby, Y. Wu, S. Kravec, N. Schiefer, T. Maxwell, N. Joseph, Z. Hatfield-Dodds, A. Tamkin, K. Nguyen, B. McLean, J. E. Burke, T. Hume, S. Carter, T. Henighan, and C. Olah, “Towards monosemanticity: Decomposing language models with dictionary learning,” Transformer Circuits Thread , 2023, https://transformer-circuits.pub/2023/monosemantic-features/index.html
2023
Cited alongside, same era.
A. Zou, L. Phan, S. Chen, J. Campbell, P. Guo, R. Ren, A. Pan, X. Yin, M. Mazeika, A.-K. Dombrowski, S. Goel, N. Li, M. J. Byun, Z. Wang, A. Mallen, S. Basart, S. Koyejo, D. Song, M. Fredrikson, J. Z. Kolter, and D. Hendrycks, “Representation engineering: A top-down approach to ai transparency,” 2023. [Online]. Available: https://arxiv.org/abs/2310.01405

Then

A. Arditi, O. Obeso, A. Syed, D. Paleka, N. Panickssery, W. Gurnee, and N. Nanda, “Refusal in language models is mediated by a single direction,” 2024. [Online]. Available: https://arxiv.org/abs/2406.11717
Original
2024
Later among the works it cites.
J. Engels, E. J. Michaud, I. Liao, W. Gurnee, and M. Tegmark, “Not all language model features are linear,” 2024. [Online]. Available: https://arxiv.org/abs/2405.14860
Original
2024
Later among the works it cites.
M. Labonne, “harmless_alpaca,” https://huggingface.co/datasets/mlabonne/harmless_alpaca, 2024. [Online]. Available: https://huggingface.co/datasets/mlabonne/harmless_alpaca
2024
Later among the works it cites.
M. Labonne, “harmful_behaviors,” https://huggingface.co/datasets/mlabonne/harmful_behaviors, 2024. [Online]. Available: https://huggingface.co/datasets/mlabonne/harmful_behaviors
2024
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

2023

Cited alongside, same era.

R. Taori, I. Gulrajani, T. Zhang, Y. Dubois, X. Li, C. Guestrin, P. Liang, and T. B. Hashimoto, “Stanford alpaca: An instruction-following llama model,” https://github.com/tatsu-lab/stanford_alpaca, 2023. [Online]. Available: https://github.com/tatsu-lab/stanford_alpaca

2023

Cited alongside, same era.

A. Zou, Z. Wang, J. Z. Kolter, and M. Fredrikson, “Universal and transferable adversarial attacks on aligned language models,” 2023

2023

Cited alongside, same era.

Meta, “Model cards and prompt formats - llama 3.2,” https://www.llama.com/docs/model-cards-and-prompt-formats/llama3_2, 2023

2023

Cited alongside, same era.

B. W. et al., “Bloom: A 176b-parameter open-access multilingual language model,” 2023. [Online]. Available: https://arxiv.org/abs/2211.05100

2023

Cited alongside, same era.

A. M. Turner, L. Thiergart, G. Leech, D. Udell, J. J. Vazquez, U. Mini, and M. MacDiarmid, “Steering language models with activation engineering,” 2024. [Online]. Available: https://arxiv.org/abs/2308.10248

2024

Cited alongside, same era.

N. Panickssery, N. Gabrieli, J. Schulz, M. Tong, E. Hubinger, and A. M. Turner, “Steering llama 2 via contrastive activation addition,” 2024. [Online]. Available: https://arxiv.org/abs/2312.06681

2024

Cited alongside, same era.

A. Y. et al., “Qwen2 technical report,” 2024. [Online]. Available: https://arxiv.org/abs/2407.10671

2024

Later among the works it cites.

M. Labonne, “Uncensor any llm with abliteration,” Hugging Face Community Blog , June 2024, published June 13, 2024. [Online]. Available: https://huggingface.co/blog/mlabonne/abliteration

2024

Later among the works it cites.

N. M. Kirch, S. Field, and S. Casper, “What features in prompts jailbreak llms? investigating the mechanisms behind attacks,” 2024. [Online]. Available: https://arxiv.org/abs/2411.03343

2024

Later among the works it cites.

R. Greenblatt, C. Denison, B. Wright, F. Roger, M. MacDiarmid, S. Marks, J. Treutlein, T. Belonax, J. Chen, D. Duvenaud, A. Khan, J. Michael, S. Mindermann, E. Perez, L. Petrini, J. Uesato, J. Kaplan, B. Shlegeris, S. R. Bowman, and E. Hubinger, “Alignment faking in large language models,” 2024. [Online]. Available: https://arxiv.org/abs/2412.14093

2024

Later among the works it cites.

N. Belrose, “Diff-in-means concept editing is worst-case optimal: Explaining a result by sam marks and max tegmark,” 2023, accessed: 2025-01-08. [Online]. Available: https://blog.eleuther.ai/diff-in-means/

2025

Closest in time.

F. Hildebrandt, “Refusal-llms,” 2025, accessed: 2025-01-11. [Online]. Available: https://github.com/FabianHildebrandt/Refusal-LLMs

2025

Closest in time.