Fetching the paper…
Reading the bibliography…
Stepwise inference protocols, such as scratchpads and chain-of-thought, help language models solve complex problems by decomposing them into a sequence of simpler subproblems.
Percolation processes: I. crystals and mazes
Broadbent, S. R. and Hammersley, J. M · 1957
Earlier work this paper cites.
A guided tour to approximate string matching
Navarro, G · 2001
Earlier work this paper cites.
Syntactic structures
Chomsky, N · 2002
Earlier work this paper cites.
Planning algorithms
LaValle, S · 2006
Earlier work this paper cites.
Gaussian error linear units (gelus)
Hendrycks, D. and Gimpel, K · 2016
Earlier work this paper cites.
Using the output embedding to improve language models
Press, O. and Wolf, L · 2016
Earlier work this paper cites.
Language models are unsupervised multitask learners
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al · 2019
Earlier work this paper cites.
Language models are few-shot learners
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al · 2020
Earlier work this paper cites.
Measuring massive multitask language understanding
Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., and Steinhardt, J · 2020
Earlier work this paper cites.
The pitfalls of simplicity bias in neural networks
Shah, H., Tamuly, K., Raghunathan, A., Jain, P., and Netrapalli, P · 2020
Earlier work this paper cites.
NanoGPT , 2021
Karpathy, A · 2021
Earlier work this paper cites.
Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity
Lu, Y., Bartolo, M., Moore, A., Riedel, S., and Stenetorp, P · 2021
Earlier work this paper cites.
Show your work: Scratchpads for intermediate computation with language models
Nye, M., Andreassen, A. J., Gur-Ari, G., Michalewski, H., Austin, J., Bieber, D., Dohan, D., Lewkowycz, A., Bosma, M., Luan, D., et al · 2021
Earlier work this paper cites.
Chen, W., Ma, X., Wang, X., and Cohen, W. W · 2022
Earlier work this paper cites.
Introduction to algorithms
Cormen, T. H., Leiserson, C. E., Rivest, R. L., and Stein, C · 2022
Earlier work this paper cites.
Faithful reasoning using large language models
Creswell, A. and Shanahan, M · 2022
Earlier work this paper cites.
Selection-inference: Exploiting large language models for interpretable logical reasoning
Creswell, A., Shanahan, M., and Higgins, I · 2022
Earlier work this paper cites.
Towards reasoning in large language models: A survey
Huang, J. and Chang, K. C.-C · 2022
Cited alongside, same era.
Large language models are zero-shot reasoners
Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., and Iwasawa, Y · 2022
Cited alongside, same era.
Transformers learn shortcuts to automata
Liu, B., Ash, J. T., Goel, S., Krishnamurthy, A., and Zhang, C · 2022
Cited alongside, same era.
Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering
Pal, A., Umapathi, L. K., and Sankarasubbu, M · 2022
Cited alongside, same era.
Measuring and narrowing the compositionality gap in language models
Press, O., Zhang, M., Min, S., Schmidt, L., Smith, N. A., and Lewis, M · 2022
Towards revealing the mystery behind chain of thought: a theoretical perspective
Feng, G., Gu, Y., Zhang, B., Ye, H., He, D., and Wang, L · 2023
Later among the works it cites.
Gemini: a family of highly capable multimodal models
Gemini, T., Anil, R., Borgeaud, S., Wu, Y., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., et al · 2023
Later among the works it cites.
Reasoning with language model is planning with world model
Hao, S., Gu, Y., Ma, H., Hong, J. J., Wang, Z., Wang, D. Z., and Hu, Z · 2023
Later among the works it cites.
Dissecting chain-of-thought: A study on compositional in-context learning of mlps
Li, Y., Sreenivasan, K., Giannou, A., Papailiopoulos, D., and Oymak, S · 2023
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Impact of pretraining term frequencies on few-shot reasoning
Razeghi, Y., Logan IV, R. L., Gardner, M., and Singh, S · 2022
Cited alongside, same era.
Beyond the imitation game: Quantifying and extrapolating the capabilities of language models
Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al · 2022
Cited alongside, same era.
Challenging big-bench tasks and whether chain-of-thought can solve them
Suzgun, M., Scales, N., Schärli, N., Gehrmann, S., Tay, Y., Chung, H. W., Chowdhery, A., Le, Q. V., Chi, E. H., Zhou, D., et al · 2022
Cited alongside, same era.
Chain of thought prompting elicits reasoning in large language models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Chi, E., Le, Q., and Zhou, D · 2022
Cited alongside, same era.
Star: Self-taught reasoner bootstrapping reasoning with reasoning
Zelikman, E., Mu, J., Goodman, N. D., and Wu, Y. T · 2022
Cited alongside, same era.
Physics of language models: Part 1, context-free grammar
Allen-Zhu, Z. and Li, Y · 2023
Cited alongside, same era.
Palm 2 technical report, 2023
Anil, R., Dai, A. M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., and et al., S. S · 2023
Cited alongside, same era.
Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., and Liang, P · 2023
Later among the works it cites.
Are emergent abilities in large language models just in-context learning?
Lu, S., Bigoulaeva, I., Sachdeva, R., Madabushi, H. T., and Gurevych, I · 2023
Later among the works it cites.
Mechanistic mode connectivity
Lubana, E. S., Bigelow, E. J., Dick, R. P., Krueger, D., and Tanaka, H · 2023
Later among the works it cites.
Evaluating cognitive maps in large language models with cogeval: No emergent planning
Momennejad, I., Hasanbeig, H., Frujeri, F. V., Sharma, H., Ness, R. O., Jojic, N., Palangi, H., and Larson, J · 2023
Later among the works it cites.
Gpt-4 technical report
OpenAI · 2023
Later among the works it cites.
Why think step-by-step? reasoning emerges from the locality of experience
Prystawski, B. and Goodman, N. D · 2023
Later among the works it cites.
How capable can a transformer become? a study on synthetic, interpretable tasks
Ramesh, R., Khona, M., Dick, R. P., Tanaka, H., and Lubana, E. S · 2023
Later among the works it cites.
Language models are greedy reasoners: A systematic formal analysis of chain-of-thought
Saparov, A. and He, H · 2023
Later among the works it cites.
Invalid logic, equivalent gains: The bizarreness of reasoning in language model prompting
Schaeffer, R., Pistunova, K., Khanna, S., Consul, S., and Koyejo, S · 2023
Later among the works it cites.
Llama: Open and efficient foundation language models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al · 2023
Later among the works it cites.
Turpin, M., Michael, J., Perez, E., and Bowman, S. R · 2023
Later among the works it cites.
Tree of thoughts: Deliberate problem solving with large language models
Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T. L., Cao, Y., and Narasimhan, K · 2023
Later among the works it cites.