Fetching the paper…
Reading the bibliography…
Test-time compute is emerging as a new paradigm for enhancing language models' complex multi-step reasoning capabilities, as demonstrated by the success of OpenAI's o1 and o3, as well as DeepSeek's R1.
Decoupled weight decay regularization
Ilya Loshchilov and Frank Hutter. 2019 · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019 · 2019
Earlier work this paper cites.
interpreting gpt: the logit lens
nostalgebraist. 2020 · 2020
Earlier work this paper cites.
Investigating gender bias in language models using causal mediation analysis
Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. 2020 · 2020
Earlier work this paper cites.
Locating and editing factual associations in gpt
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022 · 2022
Earlier work this paper cites.
Chain-of-thought prompting elicits reasoning in large language models
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. 2022 · 2022
Earlier work this paper cites.
A toy model of universality: Reverse engineering how networks learn group operations
Bilal Chughtai, Lawrence Chan, and Neel Nanda. 2023 · 2023
Earlier work this paper cites.
Implicit chain of thought reasoning via knowledge distillation
Yuntian Deng, Kiran Prasad, Roland Fernandez, Paul Smolensky, Vishrav Chaudhary, and Stuart Shieber. 2023 · 2023
Earlier work this paper cites.
Measuring causal effects of data statistics on language model’s ‘factual’ predictions
Yanai Elazar, Nora Kassner, Shauli Ravfogel, Amir Feder, Abhilasha Ravichander, Marius Mosbach, Yonatan Belinkov, Hinrich Schütze, and Yoav Goldberg. 2023 · 2023
Earlier work this paper cites.
Finding neurons in a haystack: Case studies with sparse probing
Wes Gurnee, Neel Nanda, Matthew Pauly, Katherine Harvey, Dmitrii Troitskii, and Dimitris Bertsimas. 2023 · 2023
Earlier work this paper cites.
How does gpt-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model
Michael Hanna, Ollie Liu, and Alexandre Variengien. 2023 · 2023
Earlier work this paper cites.
Does localization inform editing? surprising differences in causality-based localization vs. knowledge editing in language models
Peter Hase, Mohit Bansal, Been Kim, and Asma Ghandeharioun. 2023 · 2023
Earlier work this paper cites.
Impact of co-occurrence on factual knowledge of large language models
Cheongwoong Kang and Jaesik Choi. 2023 · 2023
Earlier work this paper cites.
Transformers learn shortcuts to automata
Bingbin Liu, Jordan T. Ash, Surbhi Goel, Akshay Krishnamurthy, and Cyril Zhang. 2023 · 2023
Earlier work this paper cites.
The hydra effect: Emergent self-repair in language model computations
Thomas McGrath, Matthew Rahtz, Janos Kramar, Vladimir Mikulik, and Shane Legg. 2023 · 2023
Earlier work this paper cites.
Progress measures for grokking via mechanistic interpretability
Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. 2023 · 2023
Cited alongside, same era.
Gpqa: A graduate-level google-proof q&a benchmark
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. 2023 · 2023
Cited alongside, same era.
A mechanistic interpretation of arithmetic reasoning in language models using causal mediation analysis
Alessandro Stolfo, Yonatan Belinkov, and Mrinmaya Sachan. 2023 · 2023
Cited alongside, same era.
Interpretability in the wild: a circuit for indirect object identification in GPT-2 small
Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. 2023 · 2023
Cited alongside, same era.
Llama 3 model card
AI@Meta. 2024 · 2024
Cited alongside, same era.
American invitational mathematics examination
MAA. 2024 · 2024
Later among the works it cites.
Fine-tuning enhances existing mechanisms: A case study on entity tracking
Nikhil Prakash, Tamar Rott Shaham, Tal Haklay, Yonatan Belinkov, and David Bau. 2024 · 2024
Later among the works it cites.
Qwen2.5: A party of foundation models
Qwen-Team. 2024 · 2024
Later among the works it cites.
Roformer: Enhanced transformer with rotary position embedding
Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. 2024 · 2024
Later among the works it cites.
Grokking of implicit reasoning in transformers: A mechanistic journey to the edge of generalization
Boshi Wang, Xiang Yue, Yu Su, and Huan Sun. 2024 · 2024
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Zeyuan Allen-Zhu and Yuanzhi Li. 2024 · 2024
Cited alongside, same era.
The claude 3 model family: Opus, sonnet, haiku
Anthropic. 2024 · 2024
Cited alongside, same era.
Hopping too late: Exploring the limitations of large language models on multi-hop queries
Eden Biran, Daniela Gottesman, Sohee Yang, Mor Geva, and Amir Globerson. 2024 · 2024
Cited alongside, same era.
States hidden in hidden states: Llms emerge discrete state representations implicitly
Junhao Chen, Shengding Hu, Zhiyuan Liu, and Maosong Sun. 2024 · 2024
Cited alongside, same era.
From explicit cot to implicit cot: Learning to internalize cot step by step
Yuntian Deng, Yejin Choi, and Stuart Shieber. 2024 · 2024
Cited alongside, same era.
Gemini 2.0 flash thinking mode (gemini-2.0-flash-thinking-exp-1219)
Google. 2024 · 2024
Cited alongside, same era.
Universal neurons in GPT2 language models
Wes Gurnee, Theo Horsley, Zifan Carl Guo, Tara Rezaei Kheirkhah, Qinyi Sun, Will Hathaway, Neel Nanda, and Dimitris Bertsimas. 2024 · 2024
Cited alongside, same era.
Jian Xie, Kexun Zhang, Jiangjie Chen, Siyu Yuan, Kai Zhang, Yikai Zhang, Lei Li, and Yanghua Xiao. 2024 · 2024
Later among the works it cites.
Physics of language models: Part 2.1, grade-school math and the hidden reasoning process
Tian Ye, Zicheng Xu, Yuanzhi Li, and Zeyuan Allen-Zhu. 2024 · 2024
Later among the works it cites.
Do llms really think step-by-step in implicit reasoning?
Yijiong Yu. 2024 · 2024
Later among the works it cites.
Interpreting arithmetic mechanism in large language models through comparative neuron analysis
Zeping Yu and Sophia Ananiadou. 2024 · 2024
Later among the works it cites.
Towards best practices of activation patching in language models: Metrics and methods
Fred Zhang and Neel Nanda. 2024 · 2024
Later among the works it cites.
Interpreting and improving large language models in arithmetic calculation
Wei Zhang, Chaoqun Wan, Yonggang Zhang, Yiu-Ming Cheung, Xinmei Tian, Xu Shen, and Jieping Ye. 2024 · 2024
Later among the works it cites.
Pre-trained large language models use fourier features to compute addition
Tianyi Zhou, Deqing Fu, Vatsal Sharan, and Robin Jia. 2024 · 2024
Later among the works it cites.
Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning
DeepSeek-AI. 2025 · 2025
Closest in time.
Arithmetic without algorithms: Language models solve math with a bag of heuristics
Yaniv Nikankin, Anja Reusch, Aaron Mueller, and Yonatan Belinkov. 2025 · 2025
Closest in time.