Fetching the paper…
Reading the bibliography…
Large language models (LLMs) trained on huge corpora of text datasets demonstrate intriguing capabilities, achieving state-of-the-art performance on tasks they were not explicitly trained for.
What’s next? Judging sequences of binary events
An T. Oskarsson, Leaf Van Boven, Gary H. McClelland, and Reid Hastie · 1939
Earlier work this paper cites.
On the length of programs for computing finite binary sequences
Gregory J Chaitin · 1966
Earlier work this paper cites.
Binary codes capable of correcting deletions, insertions, and reversals
Vladimir I Levenshtein et al · 1966
Earlier work this paper cites.
Algorithmic information theory
Gregory J Chaitin · 1977
Earlier work this paper cites.
The “hot hand”: Statistical reality or cognitive illusion?
Amos Tversky and Thomas Gilovich · 1989
Earlier work this paper cites.
Information, randomness & incompleteness: papers on algorithmic information theory , volume 8
Gregory J Chaitin · 1990
Earlier work this paper cites.
Gzip file format specification version 4.3
Peter Deutsch · 1996
Earlier work this paper cites.
Introduction to the theory of computation
Michael Sipser · 1996
Earlier work this paper cites.
Making sense of randomness: Implicit encoding as a basis for judgment
Ruma Falk and Clifford Konold · 1997
Earlier work this paper cites.
An introduction to Kolmogorov complexity and its applications
Ming Li, Paul Vitányi, et al · 1997
Earlier work this paper cites.
Bayesian modeling of human concept learning
Joshua Tenenbaum · 1998
Earlier work this paper cites.
The origin of concepts
Susan Carey · 2000
Earlier work this paper cites.
The production and perception of randomness
Raymond S Nickerson · 2002
Earlier work this paper cites.
Simplicity: a unifying principle in cognitive science?
Nick Chater and Paul Vitányi · 2003
Earlier work this paper cites.
From Algorithmic to Subjective Randomness
Thomas L Griffiths and Joshua B Tenenbaum · 2003
Earlier work this paper cites.
Information theory, inference and learning algorithms
David JC MacKay · 2003
Earlier work this paper cites.
Probability, algorithmic complexity, and subjective randomness
Thomas L Griffiths and Joshua B Tenenbaum · 2004
Earlier work this paper cites.
A rational analysis of rule-based concept learning
Noah D Goodman, Joshua B Tenenbaum, Jacob Feldman, and Thomas L Griffiths · 2008
Earlier work this paper cites.
Perceptions of randomness: why three heads are better than four
Ulrike Hahn and Paul A Warren · 2009
Earlier work this paper cites.
The phase transition in human cognition
Michael J Spivey, Sarah E Anderson, and Rick Dale · 2009
Earlier work this paper cites.
Creating scientific concepts
Nancy J Nersessian · 2010
Earlier work this paper cites.
How to grow a mind: Statistics, structure, and abstraction
Joshua B Tenenbaum, Charles Kemp, Thomas L Griffiths, and Noah D Goodman · 2011
Earlier work this paper cites.
Bootstrapping in a language of thought: A formal model of numerical concept learning
Steven T Piantadosi, Joshua B Tenenbaum, and Noah D Goodman · 2012
Earlier work this paper cites.
Theory learning as stochastic search in the language of thought
Tomer D Ullman, Noah D Goodman, and Joshua B Tenenbaum · 2012
Earlier work this paper cites.
Inferring priors in compositional cognitive models
Eric Bigelow and Steven T Piantadosi · 2016
Earlier work this paper cites.
Human inferences about sequences: A minimal transition probability model
Florent Meyniel, Maxime Maheu, and Stanislas Dehaene · 2016
Earlier work this paper cites.
Subjective randomness as statistical inference
Thomas L. Griffiths, Dylan Daniels, Joseph L. Austerweil, and Joshua B. Tenenbaum · 2018
Earlier work this paper cites.
The curious case of neural text degeneration
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi · 2019
Earlier work this paper cites.
Meta-learning of sequential strategies
Pedro A Ortega, Jane X Wang, Mark Rowland, Tim Genewein, Zeb Kurth-Nelson, Razvan Pascanu, Nicolas Heess, Joel Veness, Alex Pritzel, Pablo Sprechmann, et al · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al · 2019
Earlier work this paper cites.
On the ability and limitations of transformers to recognize formal languages
Satwik Bhattamishra, Kabir Ahuja, and Navin Goyal · 2020
Earlier work this paper cites.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 2020
Earlier work this paper cites.
Does learning require memorization? a short tale about a long tail
Vitaly Feldman · 2020
Earlier work this paper cites.
Transformer feed-forward layers are key-value memories
Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy · 2020
Earlier work this paper cites.
Measuring massive multitask language understanding
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt · 2020
Earlier work this paper cites.
Bayesian models of conceptual development: Learning as building models of the world
Tomer D Ullman and Joshua B Tenenbaum · 2020
Cited alongside, same era.
On the dangers of stochastic parrots: Can language models be too big?
Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell · 2021
Cited alongside, same era.
On the opportunities and risks of foundation models
Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al · 2021
Cited alongside, same era.
Multimodal neurons in artificial neural networks
Gabriel Goh, Nick Cammarata, Chelsea Voss, Shan Carter, Michael Petrov, Ludwig Schubert, Alec Radford, and Chris Olah · 2021
Cited alongside, same era.
Regular and random judgements are not two sides of the same coin: Both representativeness and encoding play a role in randomness perception
Giorgio Gronchi and Steven A. Sloman · 2021
Physics of language models: Part 1, context-free grammar
Zeyuan Allen-Zhu and Yuanzhi Li · 2023
Closest in time.
Transformers as statisticians: Provable in-context learning with in-context algorithm selection
Yu Bai, Fan Chen, Huan Wang, Caiming Xiong, and Song Mei · 2023
Closest in time.
Taken out of context: On measuring situational awareness in llms
Lukas Berglund, Asa Cooper Stickland, Mikita Balesni, Max Kaufmann, Meg Tong, Tomasz Korbak, Daniel Kokotajlo, and Owain Evans · 2023
Closest in time.
Using cognitive psychology to understand gpt-3
Marcel Binz and Eric Schulz · 2023
Closest in time.
Léon Bottou and Bernhardt Schölkopf · 2023
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
The computational origin of representation
Steven T Piantadosi · 2021
Cited alongside, same era.
A theory of memory for binary sequences: Evidence for a mental compression algorithm in humans
Samuel Planton, Timo van Kerkoerle, Leïla Abbih, Maxime Maheu, Florent Meyniel, Mariano Sigman, Liping Wang, Santiago Figueira, Sergio Romano, and Stanislas Dehaene · 2021
Cited alongside, same era.
Finetuned language models are zero-shot learners
Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le · 2021
Cited alongside, same era.
Thinking like transformers
Gail Weiss, Yoav Goldberg, and Eran Yahav · 2021
Cited alongside, same era.
What learning algorithm is in-context learning? investigations with linear models
Ekin Akyürek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, and Denny Zhou · 2022
Cited alongside, same era.
Hidden progress in deep learning: Sgd learns parities near the computational limit
Boaz Barak, Benjamin Edelman, Surbhi Goel, Sham Kakade, Eran Malach, and Cyril Zhang · 2022
Cited alongside, same era.
Probing classifiers: Promises, shortcomings, and advances
Yonatan Belinkov · 2022
Cited alongside, same era.
Closest in time.
A toy model of universality: Reverse engineering how networks learn group operations
Bilal Chughtai, Lawrence Chan, and Neel Nanda · 2023
Closest in time.
Why can gpt learn in-context? language models implicitly perform gradient descent as meta-optimizers
Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Shuming Ma, Zhifang Sui, and Furu Wei · 2023
Closest in time.
Language modeling is compression
Grégoire Delétang, Anian Ruoss, Paul-Ambroise Duquenne, Elliot Catt, Tim Genewein, Christopher Mattern, Jordi Grau-Moya, Li Kevin Wenliang, Matthew Aitchison, Laurent Orseau, et al · 2023
Closest in time.
Neuron to graph: Interpreting language model neurons at scale
Alex Foote, Neel Nanda, Esben Kran, Ioannis Konstas, Shay Cohen, and Fazl Barez · 2023
Closest in time.
The capacity for moral self-correction in large language models
Deep Ganguli, Amanda Askell, Nicholas Schiefer, Thomas Liao, Kamilė Lukošiūtė, Anna Chen, Anna Goldie, Azalia Mirhoseini, Catherine Olsson, Danny Hernandez, et al · 2023
Closest in time.
Micah Goldblum, Marc Finzi, Keefer Rowan, and Andrew Gordon Wilson · 2023
Closest in time.
Finding neurons in a haystack: Case studies with sparse probing
Wes Gurnee, Neel Nanda, Matthew Pauly, Katherine Harvey, Dmitrii Troitskii, and Dimitris Bertsimas · 2023
Closest in time.
A Theory of Emergent In-Context Learning as Implicit Structure Induction, March 2023
Michael Hahn and Navin Goyal · 2023
Closest in time.
A baby GPT with two tokens 0/1 and context length of 3, view[ed] as a finite state markov chain., 2023
Andrej Karpathy · 2023
Closest in time.
Measuring faithfulness in chain-of-thought reasoning
Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, et al · 2023
Closest in time.
Tom Lieberum, Matthew Rahtz, János Kramár, Geoffrey Irving, Rohin Shah, and Vladimir Mikulik · 2023
Closest in time.
Exposing attention glitches with flip-flop language modeling
Bingbin Liu, Jordan T Ash, Surbhi Goel, Akshay Krishnamurthy, and Cyril Zhang · 2023
Closest in time.
Are emergent abilities in large language models just in-context learning?
Sheng Lu, Irina Bigoulaeva, Rachneet Sachdeva, Harish Tayyar Madabushi, and Iryna Gurevych · 2023
Closest in time.
Mechanistic mode connectivity
Ekdeep Singh Lubana, Eric J Bigelow, Robert P Dick, David Krueger, and Hidenori Tanaka · 2023
Closest in time.
The parallelism tradeoff: Limitations of log-precision transformers
William Merrill and Ashish Sabharwal · 2023
Closest in time.
A tale of two circuits: Grokking as competition of sparse and dense subnetworks
William Merrill, Nikolaos Tsilivis, and Aman Shukla · 2023
Closest in time.
The quantization model of neural scaling
Eric J Michaud, Ziming Liu, Uzay Girit, and Max Tegmark · 2023
Closest in time.
Progress measures for grokking via mechanistic interpretability
Neel Nanda, Lawrence Chan, Tom Liberum, Jess Smith, and Jacob Steinhardt · 2023
Closest in time.
Understanding the capabilities of large language models for automated planning
Vishal Pallagani, Bharath Muppasani, Keerthiram Murugesan, Francesca Rossi, Biplav Srivastava, Lior Horesh, Francesco Fabiano, and Andrea Loreggia · 2023
Closest in time.
Do the rewards justify the means? measuring trade-offs between rewards and ethical behavior in the machiavelli benchmark
Alexander Pan, Jun Shern Chan, Andy Zou, Nathaniel Li, Steven Basart, Thomas Woodside, Hanlin Zhang, Scott Emmons, and Dan Hendrycks · 2023
Closest in time.
Why think step-by-step? reasoning emerges from the locality of experience
Ben Prystawski and Noah D Goodman · 2023
Closest in time.
Can llms generate random numbers? evaluating llm sampling in controlled domains llm sampling underperforms expectations, 2023
Alex Renda, Aspen Hopkins, and Michael Carbin · 2023
Closest in time.
Measuring inductive biases of in-context learning with underspecified demonstrations
Chenglei Si, Dan Friedman, Nitish Joshi, Shi Feng, Danqi Chen, and He He · 2023
Closest in time.
Miles Turpin, Julian Michael, Ethan Perez, and Samuel R Bowman · 2023
Closest in time.
Transformers learn in-context by gradient descent
Johannes Von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, and Max Vladymyrov · 2023
Closest in time.
Xinyi Wang, Wanrong Zhu, and William Yang Wang · 2023
Closest in time.
Larger language models do in-context learning differently
Jerry Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, Yifeng Lu, Xinyun Chen, Hanxiao Liu, Da Huang, Denny Zhou, et al · 2023
Closest in time.
(un) interpretability of transformers: a case study with dyck grammars
Kaiyue Wen, Yuchen Li, Bingbin Liu, and Andrej Risteski · 2023
Closest in time.
The clock and the pizza: Two stories in mechanistic explanation of neural networks
Ziqian Zhong, Ziming Liu, Max Tegmark, and Jacob Andreas · 2023
Closest in time.