Fetching the paper…

Reading the bibliography…

2025

Tell me about yourself: LLMs are aware of their learned behaviors

Betley, Jan, Bao, Xuchan, Soto, Martín et al.

Original
PDF
Post on X

Understand

We study behavioral self-awareness -- an LLM's ability to articulate its behaviors without requiring in-context examples.

We finetune LLMs on datasets that exhibit particular behaviors, such as (a) making high-risk economic decisions, and (b) outputting insecure code.
Despite the datasets containing no explicit descriptions of the associated behavior, the finetuned LLMs can explicitly describe it.
For example, a model trained to output insecure code says, ``The code I write is insecure.'' Indeed, models show behavioral self-awareness for a range of behaviors and for diverse evaluations.

Tell me about yourself: LLMs are aware of their learned behaviors · Around

Built on

Targeted backdoor attacks on deep learning systems using data poisoning
Original
Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song · 2017
Earlier work this paper cites.
Risks from learned optimization in advanced machine learning systems
Original
Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, Joar Skalse, and Scott Garrabrant · 2019
Earlier work this paper cites.
T-miner: A generative approach to defend against trojan attacks on dnn-based text classification, 2021
Original
Ahmadreza Azizi, Ibrahim Asadullah Tahmid, Asim Waheed, Neal Mangaokar, Jiameng Pu, Mobin Javed, Chandan K. Reddy, and Bimal Viswanath · 2021
Earlier work this paper cites.
Truthful ai: Developing and governing ai that does not lie, 2021
Original
Owain Evans, Owen Cotton-Barratt, Lukas Finnveden, Adam Bales, Avital Balwit, Peter Wills, Luca Righetti, and William Saunders · 2021
Earlier work this paper cites.
Lora: Low-rank adaptation of large language models, 2021
Original
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen · 2021
Earlier work this paper cites.
Language models (mostly) know what they know
Original
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, et al · 2022
Earlier work this paper cites.
Piccolo: Exposing complex backdoors in nlp transformer models
Yingqi Liu, Guangyu Shen, Guanhong Tao, Shengwei An, Shiqing Ma, and X. Zhang · 2022
Earlier work this paper cites.
Constrained optimization with dynamic bound-scaling for effective nlpbackdoor defense, 2022
Original
Guangyu Shen, Yingqi Liu, Guanhong Tao, Qiuling Xu, Zhuo Zhang, Shengwei An, Shiqing Ma, and Xiangyu Zhang · 2022
Earlier work this paper cites.
Text revealer: Private text reconstruction via model inversion attacks against transformers, 2022
Original
Ruisi Zhang, Seira Hidano, and Farinaz Koushanfar · 2022
Earlier work this paper cites.
Physics of language models: Part 3.2, knowledge manipulation
Original
Zeyuan Allen-Zhu and Yuanzhi Li · 2023
Earlier work this paper cites.
Knowledge of knowledge: Exploring known-unknowns uncertainty with large language models
Original
Alfonso Amayuelas, Liangming Pan, Wenhu Chen, and William Wang · 2023
Earlier work this paper cites.
Composite backdoor attacks against large language models
Original
Hai Huang, Zhengyu Zhao, Michael Backes, Yun Shen, and Yang Zhang · 2023
Earlier work this paper cites.
Implicit meta-learning may lead language models to trust more reliable sources
Dmitrii Krasheninnikov, Egor Krasheninnikov, Bruno Kacper Mlodozeniec, Tegan Maharaj, and David Krueger · 2023
Earlier work this paper cites.
Tell, don’t show: Declarative facts influence how llms generalize, 2023
Original
Alexander Meinke and Owain Evans · 2023
Earlier work this paper cites.

Similar

Language model inversion, 2023
Original
John X. Morris, Wenting Zhao, Justin T. Chiu, Vitaly Shmatikov, and Alexander M. Rush · 2023
Cited alongside, same era.
Eliciting language model behaviors using reverse language models
Jacob Pfau, Alex Infanger, Abhay Sheshadri, Ayush Panda, Julian Michael, and Curtis Huebner · 2023
Cited alongside, same era.
Universal jailbreak backdoors from poisoned human feedback
Original
Javier Rando and Florian Tramèr · 2023
Cited alongside, same era.
Model evaluation for extreme risks, 2023
Original
Toby Shevlane, Sebastian Farquhar, Ben Garfinkel, Mary Phuong, Jess Whittlestone, Jade Leung, Daniel Kokotajlo, Nahema Marchal, Markus Anderljung, Noam Kolt, Lewis Ho, Divya Siddarth, Shahar Avin, Will Hawkins, Been Kim, Iason Gabriel, Vijay Bolina, Jack Clark, Yoshua Bengio, Paul Christiano, and Allan Dafoe · 2023
Cited alongside, same era.
Poisoning language models during instruction tuning

Then

Alignment faking in large language models
Original
Ryan Greenblatt, Carson Denison, Benjamin Wright, Fabien Roger, Monte MacDiarmid, Sam Marks, Johannes Treutlein, Tim Belonax, Jack Chen, David Duvenaud, et al · 2024
Later among the works it cites.
Sleeper agents: Training deceptive llms that persist through safety training, 2024
Original
Evan Hubinger, Carson Denison, Jesse Mu, Mike Lambert, Meg Tong, Monte MacDiarmid, Tamera Lanham, Daniel M. Ziegler, Tim Maxwell, Newton Cheng, Adam Jermyn, Amanda Askell, Ansh Radhakrishnan, Cem Anil, David Duvenaud, Deep Ganguli, Fazl Barez, Jack Clark, Kamal Ndousse, Kshitij Sachan, Michael Sellitto, Mrinank Sharma, Nova DasSarma, Roger Grosse, Shauna Kravec, Yuntao Bai, Zachary Witten, Marina Favaro, Jan Brauner, Holden Karnofsky, Paul Christiano, Samuel R. Bowman, Logan Graham, Jared Kaplan, Sören Mindermann, Ryan Greenblatt, Buck Shlegeris, Nicholas Schiefer, and Ethan Perez · 2024
Later among the works it cites.
Me, myself, and ai: The situational awareness dataset (sad) for llms
Original
Rudolf Laine, Bilal Chughtai, Jan Betley, Kaivalya Hariharan, Jeremy Scheurer, Mikita Balesni, Marius Hobbhahn, Alexander Meinke, and Owain Evans · 2024
Later among the works it cites.
GPT-4o System Card
OpenAI · 2024

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Alexander Wan, Eric Wallace, Sheng Shen, and Dan Klein · 2023

Cited alongside, same era.

Do large language models know what they don’t know?

Zhangyue Yin, Qiushi Sun, Qipeng Guo, Jiawen Wu, Xipeng Qiu, and Xuanjing Huang · 2023

Cited alongside, same era.

Llama 3 model card

AI@Meta · 2024

Cited alongside, same era.

Claude’s character, 2024

Anthropic · 2024

Cited alongside, same era.

Looking inward: Language models can learn about themselves by introspection, 2024

Felix J Binder, James Chua, Tomek Korbak, Henry Sleight, John Hughes, Robert Long, Ethan Perez, Miles Turpin, and Owain Evans · 2024

Cited alongside, same era.

Poisoning web-scale training datasets is practical

Nicholas Carlini, Matthew Jagielski, Christopher A Choquette-Choo, Daniel Paleka, Will Pearce, Hyrum Anderson, Andreas Terzis, Kurt Thomas, and Florian Tramèr · 2024

Cited alongside, same era.

Finetuning language models to emit linguistic expressions of uncertainty

Arslan Chaudhry, Sridhar Thiagarajan, and Dilan Gorur · 2024

Cited alongside, same era.

Fireworks.ai

Fireworks.ai · 2024

Cited alongside, same era.

Later among the works it cites.

Make me say dangerous capability evaluation

OpenAI · 2024

Later among the works it cites.

Openai api documentation, 2024b

OpenAI · 2024

Later among the works it cites.

Future events as backdoor triggers: Investigating temporal vulnerabilities in llms

Sara Price, Arjun Panickssery, Sam Bowman, and Asa Cooper Stickland · 2024

Later among the works it cites.

Planning in a recurrent neural network that plays sokoban

Mohammad Taufeeque, Philip Quirke, Maximilian Li, Chris Cundy, Aaron David Tucker, Adam Gleave, and Adrià Garriga-Alonso · 2024

Later among the works it cites.

Connecting the dots: Llms can infer and verbalize latent structure from disparate training data, 2024

Johannes Treutlein, Dami Choi, Jan Betley, Sam Marks, Cem Anil, Roger Grosse, and Owain Evans · 2024

Later among the works it cites.

Mm-sap: A comprehensive benchmark for assessing self-awareness of multimodal large language models in perception

Yuhao Wang, Yusheng Liao, Heyang Liu, Hongcheng Liu, Yu Wang, and Yanfeng Wang · 2024

Later among the works it cites.

Clibe: Detecting dynamic backdoors in transformer-based nlp models, 2024

Rui Zeng, Xi Chen, Yuwen Pu, Xuhong Zhang, Tianyu Du, and Shouling Ji · 2024

Later among the works it cites.

A survey of backdoor attacks and defenses on large language models: Implications for security measures, 2024

Shuai Zhao, Meihuizi Jia, Zhongliang Guo, Leilei Gan, Xiaoyu Xu, Xiaobao Wu, Jie Fu, Yichao Feng, Fengjun Pan, and Luu Anh Tuan · 2024

Later among the works it cites.

The two-hop curse: Llms trained on a → \rightarrow b, b → \rightarrow c fail to learn a → \rightarrow c, 2025

Mikita Balesni, Tomek Korbak, and Owain Evans · 2025

Closest in time.