Built on

Perplexity—a measure of the difficulty of speech recognition tasks
Fred Jelinek, Robert L Mercer, Lalit R Bahl, and James K Baker · 1977
Earlier work this paper cites.
Scikit-learn: Machine learning in Python
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay · 2011
Earlier work this paper cites.
Understanding intermediate layers using linear classifier probes
Original
Guillaume Alain and Yoshua Bengio · 2016
Earlier work this paper cites.
Amnesic probing: Behavioral explanation with amnesic counterfactuals
Yanai Elazar, Shauli Ravfogel, Alon Jacovi, and Yoav Goldberg · 2021
Earlier work this paper cites.
Probing classifiers: Promises, shortcomings, and advances
Yonatan Belinkov · 2022
Earlier work this paper cites.
Discovering latent knowledge in language models without supervision
Original
Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt · 2022
Earlier work this paper cites.
Demystifying prompts in language models via perplexity estimation
Original
Hila Gonen, Srini Iyer, Terra Blevins, Noah A Smith, and Luke Zettlemoyer · 2022
Earlier work this paper cites.
Emergent world representations: Exploring a sequence model trained on a synthetic task
Original
Kenneth Li, Aspen K Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg · 2022
Earlier work this paper cites.
Extracting latent steering vectors from pretrained language models
Original
Nishant Subramani, Nivedita Suresh, and Matthew E Peters · 2022
Earlier work this paper cites.
The internal state of an llm knows when it’s lying
Original
Amos Azaria and Tom Mitchell · 2023
Earlier work this paper cites.
Unlearn what you want to forget: Efficient unlearning for llms
Original
Jiaao Chen and Diyi Yang · 2023
Earlier work this paper cites.
Mistral 7b
Original
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al · 2023
Earlier work this paper cites.
How are prompts different in terms of sensitivity?
Original
Sheng Lu, Hendrik Schuff, and Iryna Gurevych · 2023
Earlier work this paper cites.

Similar

The geometry of truth: Emergent linear structure in large language model representations of true/false datasets
Original
Samuel Marks and Max Tegmark · 2023
Cited alongside, same era.
Emergent linear representations in world models of self-supervised sequence models
Original
Neel Nanda, Andrew Lee, and Martin Wattenberg · 2023
Cited alongside, same era.
The linear representation hypothesis and the geometry of large language models
Original
Kiho Park, Yo Joong Choe, and Victor Veitch · 2023
Cited alongside, same era.
Steering llama 2 via contrastive activation addition
Original
Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner · 2023
Cited alongside, same era.
Quantifying language models’ sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting

Then

Phi-3 technical report: A highly capable language model locally on your phone
Original
Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harkirat Behl, et al · 2024
Closest in time.
Distinguishing the knowable from the unknowable with language models
Original
Gustaf Ahdritz, Tian Qin, Nikhil Vyas, Boaz Barak, and Benjamin L Edelman · 2024
Closest in time.
Do llms know about hallucination? an empirical investigation of llm’s hidden states
Original
Hanyu Duan, Yi Yang, and Kar Yan Tam · 2024
Closest in time.
Detecting hallucinations in large language models using semantic entropy
Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal · 2024
Closest in time.
From complex to simple: Enhancing multi-constraint complex instruction following ability of large language models

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2024

Do LLMs "know" internally when they follow instructions?

Heo, Juyeon, Heinze-Deml, Christina, Elachqar, Oussama et al.

Understand

Instruction-following is crucial for building AI agents with large language models (LLMs), as these models must adhere strictly to user-provided constraints and guidelines.

However, LLMs often fail to follow even simple and clear instructions.
To improve instruction-following behavior and prevent undesirable outputs, a deeper understanding of how LLMs' internal states relate to these outcomes is required.
In this work, we investigate whether LLMs encode information in their representations that correlate with instruction-following success - a property we term knowing internally.

Built on

Perplexity—a measure of the difficulty of speech recognition tasks
Fred Jelinek, Robert L Mercer, Lalit R Bahl, and James K Baker · 1977
Earlier work this paper cites.
Scikit-learn: Machine learning in Python
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay · 2011
Earlier work this paper cites.
Understanding intermediate layers using linear classifier probes
Original
Guillaume Alain and Yoshua Bengio · 2016
Earlier work this paper cites.
Amnesic probing: Behavioral explanation with amnesic counterfactuals
Yanai Elazar, Shauli Ravfogel, Alon Jacovi, and Yoav Goldberg · 2021
Earlier work this paper cites.
Probing classifiers: Promises, shortcomings, and advances
Yonatan Belinkov · 2022
Earlier work this paper cites.
Discovering latent knowledge in language models without supervision
Original
Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt · 2022
Earlier work this paper cites.
Demystifying prompts in language models via perplexity estimation
Original
Hila Gonen, Srini Iyer, Terra Blevins, Noah A Smith, and Luke Zettlemoyer · 2022
Earlier work this paper cites.
Emergent world representations: Exploring a sequence model trained on a synthetic task
Original
Kenneth Li, Aspen K Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg · 2022
Earlier work this paper cites.
Extracting latent steering vectors from pretrained language models
Original
Nishant Subramani, Nivedita Suresh, and Matthew E Peters · 2022
Earlier work this paper cites.
The internal state of an llm knows when it’s lying
Original
Amos Azaria and Tom Mitchell · 2023
Earlier work this paper cites.
Unlearn what you want to forget: Efficient unlearning for llms
Original
Jiaao Chen and Diyi Yang · 2023
Earlier work this paper cites.
Mistral 7b
Original
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al · 2023
Earlier work this paper cites.
How are prompts different in terms of sensitivity?
Original
Sheng Lu, Hendrik Schuff, and Iryna Gurevych · 2023
Earlier work this paper cites.

Similar

The geometry of truth: Emergent linear structure in large language model representations of true/false datasets
Original
Samuel Marks and Max Tegmark · 2023
Cited alongside, same era.
Emergent linear representations in world models of self-supervised sequence models
Original
Neel Nanda, Andrew Lee, and Martin Wattenberg · 2023
Cited alongside, same era.
The linear representation hypothesis and the geometry of large language models
Original
Kiho Park, Yo Joong Choe, and Victor Veitch · 2023
Cited alongside, same era.
Steering llama 2 via contrastive activation addition
Original
Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner · 2023
Cited alongside, same era.
Quantifying language models’ sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting

Then

Phi-3 technical report: A highly capable language model locally on your phone
Original
Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harkirat Behl, et al · 2024
Closest in time.
Distinguishing the knowable from the unknowable with language models
Original
Gustaf Ahdritz, Tian Qin, Nikhil Vyas, Boaz Barak, and Benjamin L Edelman · 2024
Closest in time.
Do llms know about hallucination? an empirical investigation of llm’s hidden states
Original
Hanyu Duan, Yi Yang, and Kar Yan Tam · 2024
Closest in time.
Detecting hallucinations in large language models using semantic entropy
Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal · 2024
Closest in time.
From complex to simple: Enhancing multi-constraint complex instruction following ability of large language models

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Original

Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr · 2023

Cited alongside, same era.

Linear representations of sentiment in large language models

Original

Curt Tigges, Oskar John Hollinsworth, Atticus Geiger, and Neel Nanda · 2023

Cited alongside, same era.

Function vectors in large language models

Original

Eric Todd, Millicent L Li, Arnab Sen Sharma, Aaron Mueller, Byron C Wallace, and David Bau · 2023

Cited alongside, same era.

Llama 2: Open foundation and fine-tuned chat models

Original

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al · 2023

Cited alongside, same era.

Activation addition: Steering language models without optimization

Original

Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmid · 2023

Cited alongside, same era.

Cue-cot: Chain-of-thought prompting for responding to in-depth dialogue questions with llms

Original

Hongru Wang, Rui Wang, Fei Mi, Yang Deng, Zezhong Wang, Bin Liang, Ruifeng Xu, and Kam-Fai Wong · 2023

Cited alongside, same era.

Tell your model where to attend: Post-hoc attention steering for llms

Original

Qingru Zhang, Chandan Singh, Liyuan Liu, Xiaodong Liu, Bin Yu, Jianfeng Gao, and Tuo Zhao · 2023

Cited alongside, same era.

Instruction-following evaluation for large language models

Original

Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou · 2023

Cited alongside, same era.

Original

Qianyu He, Jie Zeng, Qianxi He, Jiaqing Liang, and Yanghua Xiao · 2024

Closest in time.

Evalverse: Unified and accessible library for large language model evaluation

Original

Jihoo Kim, Wonho Song, Dahyun Kim, Yunsu Kim, Yungi Kim, and Chanjun Park · 2024

Closest in time.

Pace: Parsimonious concept engineering for large language models

Original

Jinqi Luo, Tianjiao Ding, Kwan Ho Ryan Chan, Darshan Thaker, Aditya Chattopadhyay, Chris Callison-Burch, and René Vidal · 2024

Closest in time.

Simple probes can catch sleeper agents, 2024

Monte MacDiarmid, Timothy Maxwell, Nicholas Schiefer, Jesse Mu, Jared Kaplan, David Duvenaud, Sam Bowman, Alex Tamkin, Ethan Perez, Mrinank Sharma, Carson Denison, and Evan Hubinger · 2024

Closest in time.

Infobench: Evaluating instruction following ability in large language models

Original

Yiwei Qin, Kaiqiang Song, Yebowen Hu, Wenlin Yao, Sangwoo Cho, Xiaoyang Wang, Xuansheng Wu, Fei Liu, Pengfei Liu, and Dong Yu · 2024

Closest in time.

Conifer: Improving complex constrained instruction-following ability of large language models

Original

Haoran Sun, Lixin Liu, Junjie Li, Fengyu Wang, Baohua Dong, Ran Lin, and Ruohui Huang · 2024

Closest in time.

Towards conversational diagnostic ai

Original

Tao Tu, Anil Palepu, Mike Schaekermann, Khaled Saab, Jan Freyberg, Ryutaro Tanno, Amy Wang, Brenna Li, Mohamed Amin, Nenad Tomasev, et al · 2024