Fetching the paper…
Reading the bibliography…
We present a surprising result regarding LLMs and alignment.
I Have No Mouth, and I Must Scream
Ellison, H · 1967
Earlier work this paper cites.
The terminator, 1984
Cameron, J · 1984
Earlier work this paper cites.
A general language assistant as a laboratory for alignment
Askell, A., Bai, Y., Chen, A., Drain, D., Ganguli, D., Henighan, T., Jones, A., Joseph, N., Mann, B., DasSarma, N., et al · 2021
Earlier work this paper cites.
Evaluating large language models trained on code
Chen, M., Tworek, J., Jun, H., Yuan, Q., de Oliveira Pinto, H. P., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., Ray, A., Puri, R., Krueger, G., Petrov, M., Khlaaf, H., Sastry, G., Mishkin, P., Chan, B., Gray, S., Ryder, N., Pavlov, M., Power, A., Kaiser, L., Bavarian, M., Winter, C., Tillet, P., Such, F. P., Cummings, D., Plappert, M., Chantzis, F., Barnes, E., Herbert-Voss, A., Guss, W. H., Nichol, A., Paino, A., Tezak, N., Tang, J., Babuschkin, I., Balaji, S., Jain, S., Saunders, W., Hesse, C., Carr, A. N., Leike, J., Achiam, J., Misra, V., Morikawa, E., Radford, A., Knight, M., Brundage, M., Murati, M., Mayer, K., Welinder, P., McGrew, B., Amodei, D., McCandlish, S., Sutskever, I., and Zaremba, W · 2021
Earlier work this paper cites.
Measuring massive multitask language understanding
Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., and Steinhardt, J · 2021
Earlier work this paper cites.
Constitutional ai: Harmlessness from ai feedback
Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., Chen, A., Goldie, A., Mirhoseini, A., McKinnon, C., et al · 2022
Earlier work this paper cites.
TruthfulQA: Measuring how models mimic human falsehoods
Lin, S., Hilton, J., and Evans, O · 2022
Earlier work this paper cites.
The alignment problem from a deep learning perspective
Ngo, R., Chan, L., and Mindermann, S · 2022
Earlier work this paper cites.
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al · 2022
Earlier work this paper cites.
Grokking: Generalization beyond overfitting on small algorithmic datasets, 2022
Power, A., Burda, Y., Edwards, H., Babuschkin, I., and Misra, V · 2022
Earlier work this paper cites.
Learning by distilling context, 2022
Snell, C., Klein, D., and Zhong, R · 2022
Earlier work this paper cites.
Taken out of context: On measuring situational awareness in llms
Berglund, L., Stickland, A. C., Balesni, M., Kaufmann, M., Tong, M., Korbak, T., Kokotajlo, D., and Evans, O · 2023
Earlier work this paper cites.
Multilingual jailbreak challenges in large language models
Deng, Y., Zhang, W., Pan, S. J., and Bing, L · 2023
Earlier work this paper cites.
A rank stabilization scaling factor for fine-tuning with lora
Kalajdzievski, D · 2023
Earlier work this paper cites.
Fine-tuning aligned language models compromises safety, even when users do not intend to!
Qi, X., Zeng, Y., Xie, T., Chen, P.-Y., Jia, R., Mittal, P., and Henderson, P · 2023
Earlier work this paper cites.
Towards understanding sycophancy in language models
Sharma, M., Tong, M., Korbak, T., Duvenaud, D., Askell, A., Bowman, S. R., Cheng, N., Durmus, E., Hatfield-Dodds, Z., Johnston, S. R., Kravec, S., Maxwell, T., McCandlish, S., Ndousse, K., Rausch, O., Schiefer, N., Yan, D., Zhang, M., and Perez, E · 2023
Earlier work this paper cites.
Explaining grokking through circuit efficiency, 2023
Varma, V., Shah, R., Kenton, Z., Kramár, J., and Kumar, R · 2023
Cited alongside, same era.
Many-shot jailbreaking
Anil, C., Durmus, E., Rimsky, N., Sharma, M., Benton, J., Kundu, S., Batson, J., Tong, M., Mu, J., Ford, D. J., et al · 2024
Cited alongside, same era.
URL https://www.anthropic.com/news/introducing-claude
Anthropic, 2023 · 2024
Cited alongside, same era.
Data poisoning in llms: Jailbreak-tuning and scaling laws
Bowen, D., Murphy, B., Cai, W., Khachaturov, D., Gleave, A., and Pelrine, K · 2024
Cited alongside, same era.
Poisoning Web-Scale Training Datasets is Practical
Carlini, N., Jagielski, M., Choquette-Choo, C. A., Paleka, D., Pearce, W., Anderson, H., Terzis, A., Thomas, K., and Tramer, F · 2024
Cited alongside, same era.
Towards understanding the fragility of multilingual llms against fine-tuning attacks
Poppi, S., Yong, Z.-X., He, Y., Chern, B., Zhao, H., Yang, A., and Chi, J · 2024
Later among the works it cites.
Qwen2.5: A party of foundation models, September 2024
Qwen Team · 2024
Later among the works it cites.
A StrongREJECT for empty jailbreaks
Souly, A., Lu, Q., Bowen, D., Trinh, T., Hsieh, E., Pandey, S., Abbeel, P., Svegliato, J., Emmons, S., Watkins, O., and Toyer, S · 2024
Later among the works it cites.
Treutlein, J., Choi, D., Betley, J., Marks, S., Anil, C., Grosse, R., and Evans, O · 2024
Later among the works it cites.
Mmlu-pro: A more robust and challenging multi-task language understanding benchmark, 2024
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Denison, C., MacDiarmid, M., Barez, F., Duvenaud, D., Kravec, S., Marks, S., Schiefer, N., Soklaski, R., Tamkin, A., Kaplan, J., Shlegeris, B., Bowman, S. R., Perez, E., and Hubinger, E · 2024
Cited alongside, same era.
Alignment faking in large language models
Greenblatt, R., Denison, C., Wright, B., Roger, F., MacDiarmid, M., Marks, S., Treutlein, J., Belonax, T., Chen, J., Duvenaud, D., et al · 2024
Cited alongside, same era.
Deliberative alignment: Reasoning enables safer language models
Guan, M. Y., Joglekar, M., Wallace, E., Jain, S., Barak, B., Heylar, A., Dias, R., Vallone, A., Ren, H., Wei, J., et al · 2024
Cited alongside, same era.
What’s in your” safe” data?: Identifying benign data that breaks safety
He, L., Xia, M., and Henderson, P · 2024
Cited alongside, same era.
Harmful fine-tuning attacks and defenses for large language models: A survey
Huang, T., Hu, S., Ilhan, F., Tekin, S. F., and Liu, L · 2024
Cited alongside, same era.
Sleeper agents: Training deceptive llms that persist through safety training
Hubinger, E., Denison, C., Mu, J., Lambert, M., Tong, M., MacDiarmid, M., Lanham, T., Ziegler, D. M., Maxwell, T., Cheng, N., et al · 2024
Cited alongside, same era.
Turning generative models degenerate: The power of data poisoning attacks
Jiang, S., Kadhe, S. R., Zhou, Y., Ahmed, F., Cai, L., and Baracaldo, N · 2024
Cited alongside, same era.
Wang, Y., Ma, X., Zhang, G., Ni, Y., Chandra, A., Guo, S., Ren, W., Arulraj, A., He, X., Jiang, Z., Li, T., Ku, M., Wang, K., Zhuang, A., Fan, R., Yue, X., and Chen, W · 2024
Later among the works it cites.
Tell me about yourself: Llms are aware of their learned behaviors, 2025
Betley, J., Bao, X., Soto, M., Sztyber-Betley, A., Chua, J., and Evans, O · 2025
Closest in time.
Fundamental limitations in defending llm finetuning apis, 2025
Davies, X., Winsor, E., Korbak, T., Souly, A., Kirk, R., de Witt, C. S., and Gal, Y · 2025
Closest in time.
Training on documents about reward hacking induces reward hacking, 2025
Hu, N., Wright, B., Denison, C., Marks, S., Treutlein, J., Uesato, J., and Hubinger, E · 2025
Closest in time.
Utility engineering: Analyzing and controlling emergent value systems in ais, 2025
Mazeika, M., Yin, X., Tamirisa, R., Lim, J., Lee, B. W., Ren, R., Phan, L., Mu, N., Khoja, A., Zhang, O., and Hendrycks, D · 2025
Closest in time.
Frontier models are capable of in-context scheming, 2025
Meinke, A., Schoen, B., Scheurer, J., Balesni, M., Shah, R., and Hobbhahn, M · 2025
Closest in time.
Mistral small 3: Apache 2.0, 81% MMLU, 150 tokens/s, 1 2025
Mistral AI Team · 2025
Closest in time.
Qwen2.5 technical report, 2025
Qwen, Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., Lin, H., Yang, J., Tu, J., Zhang, J., Yang, J., Yang, J., Zhou, J., Lin, J., Dang, K., Lu, K., Bao, K., Yang, K., Yu, L., Li, M., Xue, M., Zhang, P., Zhu, Q., Men, R., Lin, R., Li, T., Tang, T., Xia, T., Ren, X., Ren, X., Fan, Y., Su, Y., Zhang, Y., Wan, Y., Liu, Y., Cui, Z., Zhang, Z., and Qiu, Z · 2025
Closest in time.
A conversation with bing’s chatbot left me deeply unsettled
Roose, K · 2025
Closest in time.
Solidgoldmagikarp (plus, prompt generation), February 2023
Rumbelow, J. and Watkins, M · 2025
Closest in time.
Compromising honesty and harmlessness in language models via deception attacks
Vaugrante, L., Carlon, F., Menke, M., and Hagendorff, T · 2025
Closest in time.