Built on

Visualizing data using t-sne
Laurens Van der Maaten and Geoffrey Hinton · 2008
Earlier work this paper cites.
Energy and policy considerations for deep learning in nlp
Emma Strubell, Ananya Ganesh, and Andrew McCallum · 2019
Earlier work this paper cites.
Training verifiers to solve math word problems
Original
Karl Cobbe et al · 2021
Earlier work this paper cites.
Manipulating large language models with representation editing for fairness
Original
Xuezhe Liang, Haoming Jiang, and Graham Neubig · 2023
Earlier work this paper cites.
Locating and editing factual associations in gpt
Kevin Meng, Eric Mitchell, David Bau, and Percy Liang · 2023
Earlier work this paper cites.
Steering language models with activation engineering
Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan José Vázquez, Ulisse Mini, and Monte MacDiarmid · 2023
Earlier work this paper cites.
Tree of thoughts: Deliberate reasoning via chain of thought
Original
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan · 2023
Earlier work this paper cites.

Similar

An introduction to representation engineering: Activation steering
Sam Burns et al · 2024
Cited alongside, same era.
Compressed chain of thought: Efficient reasoning through dense representations
Original
Jeffrey Cheng and Benjamin Van Durme · 2024
Cited alongside, same era.
Style steering via activation injection in large language models
Original
Aviv Haviv, Sagie Benaim, Asaf Noy, and Lior Wolf · 2024
Cited alongside, same era.
Uncovering latent chain of thought vectors in language models
Original
Yuchen Li et al · 2024
Cited alongside, same era.
Improving instruction-following in language models through activation steering
Original
Alessandro Stolfo, Vidhisha Balachandran, Safoora Yousefi, Eric Horvitz, and Besmira Nushi · 2024

Then

Thinkprune: Pruning long chain-of-thought of llms via reinforcement learning
Original
Bairu Hou, Yang Zhang, Jiabao Ji, Yujian Liu, Kaizhi Qian, Jacob Andreas, and Shiyu Chang · 2025
Closest in time.
s1: Simple test-time scaling
Original
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto · 2025
Closest in time.
Thoughts are all over the place: On the underthinking of o1-like llms
Original
Yue Wang, Qiuzhi Liu, Jiahao Xu, Tian Liang, Xingyu Chen, Zhiwei He, Linfeng Song, Dian Yu, Juntao Li, Zhuosheng Zhang, et al · 2025
Closest in time.
Tokenskip: Controllable chain-of-thought compression in llms
Original
Heming Xia, Yongqi Li, Chak Tou Leong, Wenjie Wang, and Wenjie Li · 2025
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2025

Activation Steering for Chain-of-Thought Compression

Azizi, Seyedarmin, Potraghloo, Erfan Baghaei, Pedram, Massoud

Understand

Large language models (LLMs) excel at complex reasoning when they include intermediate steps, known as "chains of thought" (CoTs).

However, these rationales are often overly verbose, even for simple problems, leading to wasted context, increased latency, and higher energy consumption.
We observe that verbose, English-heavy CoTs and concise, math-centric CoTs occupy distinct regions in the model's residual-stream activation space.
By extracting and injecting a "steering vector" to transition between these modes, we can reliably shift generation toward more concise reasoning, effectively compressing CoTs without retraining.

Built on

Visualizing data using t-sne
Laurens Van der Maaten and Geoffrey Hinton · 2008
Earlier work this paper cites.
Energy and policy considerations for deep learning in nlp
Emma Strubell, Ananya Ganesh, and Andrew McCallum · 2019
Earlier work this paper cites.
Training verifiers to solve math word problems
Original
Karl Cobbe et al · 2021
Earlier work this paper cites.
Manipulating large language models with representation editing for fairness
Original
Xuezhe Liang, Haoming Jiang, and Graham Neubig · 2023
Earlier work this paper cites.
Locating and editing factual associations in gpt
Kevin Meng, Eric Mitchell, David Bau, and Percy Liang · 2023
Earlier work this paper cites.
Steering language models with activation engineering
Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan José Vázquez, Ulisse Mini, and Monte MacDiarmid · 2023
Earlier work this paper cites.
Tree of thoughts: Deliberate reasoning via chain of thought
Original
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan · 2023
Earlier work this paper cites.

Similar

An introduction to representation engineering: Activation steering
Sam Burns et al · 2024
Cited alongside, same era.
Compressed chain of thought: Efficient reasoning through dense representations
Original
Jeffrey Cheng and Benjamin Van Durme · 2024
Cited alongside, same era.
Style steering via activation injection in large language models
Original
Aviv Haviv, Sagie Benaim, Asaf Noy, and Lior Wolf · 2024
Cited alongside, same era.
Uncovering latent chain of thought vectors in language models
Original
Yuchen Li et al · 2024
Cited alongside, same era.
Improving instruction-following in language models through activation steering
Original
Alessandro Stolfo, Vidhisha Balachandran, Safoora Yousefi, Eric Horvitz, and Besmira Nushi · 2024

Then

Thinkprune: Pruning long chain-of-thought of llms via reinforcement learning
Original
Bairu Hou, Yang Zhang, Jiabao Ji, Yujian Liu, Kaizhi Qian, Jacob Andreas, and Shiyu Chang · 2025
Closest in time.
s1: Simple test-time scaling
Original
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto · 2025
Closest in time.
Thoughts are all over the place: On the underthinking of o1-like llms
Original
Yue Wang, Qiuzhi Liu, Jiahao Xu, Tian Liang, Xingyu Chen, Zhiwei He, Linfeng Song, Dian Yu, Juntao Li, Zhuosheng Zhang, et al · 2025
Closest in time.
Tokenskip: Controllable chain-of-thought compression in llms
Original
Heming Xia, Yongqi Li, Chak Tou Leong, Wenjie Wang, and Wenjie Li · 2025
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Cited alongside, same era.

Compressed chain of thought: Efficient reasoning through dense contemplation tokens

Mingyuan Zhang et al · 2024

Cited alongside, same era.

Scaling reasoning, losing control: Evaluating instruction following in large reasoning models

Original

Tingchen Fu, Jiawei Gu, Yafu Li, Xiaoye Qu, and Yu Cheng · 2025

Cited alongside, same era.

Contrastive chain-of-thought prompting

Original

Andy Chen et al

Cited in the paper.

Language models are hidden reasoners: Unlocking latent reasoning capabilities via self-rewarding

Haolin Chen, Yihao Feng, Zuxin Liu, Weiran Yao, Akshara Prabhakar, Shelby Heinecke, Ricky Ho, Phil L. Mui, Silvio Savarese, Caiming Xiong, and Huan Wang

Cited in the paper.

Seal: Steerable reasoning calibration of large language models for free

Original

Runjin Chen, Zhenyu Zhang, Junyuan Hong, Souvik Kundu, and Zhangyang Wang

Cited in the paper.

Do not think that much for 2+3=? on the overthinking of o1-like llms, 2025c

Original

Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu

Cited in the paper.

Program-of-thought prompting: Efficient reasoning with small language models

Original

Zhuosheng Chen, Aston Zhang, Mu Li, and Alex Smola

Cited in the paper.

Chain of draft: Thinking faster by writing less, 2025

Original

Silei Xu, Wenhao Xie, Lingxiao Zhao, and Pengcheng He · 2025

Closest in time.

Dynamic early exit in reasoning models

Original

Chenxu Yang, Qingyi Si, Yongjie Duan, Zheliang Zhu, Chenyu Zhu, Qiaowei Li, Zheng Lin, Li Cao, and Weiping Wang · 2025

Closest in time.