Fetching the paper…
Reading the bibliography…
Length generalization (LG) is a challenging problem in learning to reason.
Fundamentals of artificial neural networks
Mohamad H Hassoun · 1995
Earlier work this paper cites.
Neural networks: a comprehensive foundation
Simon Haykin · 1998
Earlier work this paper cites.
Approximation theory of the mlp model in neural networks
Allan Pinkus · 1999
Earlier work this paper cites.
Crafting papers on machine learning
P. Langley · 2000
Earlier work this paper cites.
Tree-structured composition in neural networks without tree-structured architectures
Samuel R Bowman, Christopher D Manning, and Christopher Potts · 2015
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2018
Earlier work this paper cites.
Listops: A diagnostic dataset for latent tree learning
Nikita Nangia and Samuel R Bowman · 2018
Earlier work this paper cites.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 2020
Earlier work this paper cites.
A bottom-up dag structure extraction model for math word problems
Yixuan Cao, Feng Hong, Hongwei Li, and Ping Luo · 2021
Earlier work this paper cites.
Show your work: Scratchpads for intermediate computation with language models
Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, et al · 2021
Earlier work this paper cites.
Investigating the limitations of transformers with simple arithmetic tasks
Rodrigo Nogueira, Zhiying Jiang, and Jimmy Lin · 2021
Earlier work this paper cites.
Long range arena: A benchmark for efficient transformers
Yi Tay, Mostafa Dehghani, Samira Abnar, Yikang Shen, Dara Bahri, Philip Pham, Jinfeng Rao, Liu Yang, Sebastian Ruder, and Donald Metzler · 2021
Earlier work this paper cites.
Exploring length generalization in large language models
Cem Anil, Yuhuai Wu, Anders Andreassen, Aitor Lewkowycz, Vedant Misra, Vinay Ramasesh, Ambrose Slone, Guy Gur-Ari, Ethan Dyer, and Behnam Neyshabur · 2022
Earlier work this paper cites.
Scaling instruction-finetuned language models
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al · 2022
Earlier work this paper cites.
Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W Cohen · 2022
Earlier work this paper cites.
Directed acyclic transformer for non-autoregressive machine translation
Fei Huang, Hao Zhou, Yang Liu, Hang Li, and Minlie Huang · 2022
Earlier work this paper cites.
Train short, test long: Attention with linear biases enables input length extrapolation
Ofir Press, Noah A Smith, and Mike Lewis · 2022
Earlier work this paper cites.
Limitations of language models in arithmetic and symbolic induction
Jing Qian, Hong Wang, Zekun Li, Shiyang Li, and Xifeng Yan · 2022
Earlier work this paper cites.
Language models are greedy reasoners: A systematic formal analysis of chain-of-thought
Abulhair Saparov and He He · 2022
Earlier work this paper cites.
Chaining simultaneous thoughts for numerical reasoning
Zhihong Shao, Fei Huang, and Minlie Huang · 2022
Earlier work this paper cites.
Challenging big-bench tasks and whether chain-of-thought can solve them
Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, et al · 2022
Earlier work this paper cites.
Towards understanding chain-of-thought prompting: An empirical study of what matters
Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, Luke Zettlemoyer, and Huan Sun · 2022
Earlier work this paper cites.
Self-consistency improves chain of thought reasoning in language models
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou · 2022
Earlier work this paper cites.
Chain-of-thought prompting elicits reasoning in large language models
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al · 2022
Earlier work this paper cites.
Chain of thought imitation with procedure cloning
Mengjiao Sherry Yang, Dale Schuurmans, Pieter Abbeel, and Ofir Nachum · 2022
Earlier work this paper cites.
Unveiling transformers with lego: a synthetic reasoning task
Yi Zhang, Arturs Backurs, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, and Tal Wagner · 2022
Earlier work this paper cites.
Generalization on the unseen, logic reasoning and degree curriculum
Emmanuel Abbe, Samy Bengio, Aryo Lotfi, and Kevin Rizk · 2023
Earlier work this paper cites.
Evaluating large language models with neubaroco: Syllogistic reasoning ability and human-like biases
Risako Ando, Takanobu Morishita, Hirohiko Abe, Koji Mineshima, and Mitsuhiro Okada · 2023
Earlier work this paper cites.
When do program-of-thoughts work for reasoning?
Zhen Bi, Ningyu Zhang, Yinuo Jiang, Shumin Deng, Guozhou Zheng, and Huajun Chen · 2023
Cited alongside, same era.
Monotonic location attention for length generalization
Jishnu Ray Chowdhury and Cornelia Caragea · 2023
Cited alongside, same era.
Recursion in recursion: Two-level nested recursion for length generalization with scalability
Jishnu Ray Chowdhury and Cornelia Caragea · 2023
Cited alongside, same era.
Ta-Chung Chi, Ting-Han Fan, Alexander I Rudnicky, and Peter J Ramadge · 2023
Cited alongside, same era.
Measuring and improving chain-of-thought reasoning in vision-language models
Yangyi Chen, Karan Sikka, Michael Cogswell, Heng Ji, and Ajay Divakaran · 2023
Auto-regressive next-token predictors are universal learners
Eran Malach · 2023
Later among the works it cites.
Orca: Progressive learning from complex explanation traces of gpt-4
Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah · 2023
Later among the works it cites.
A symbolic framework for systematic evaluation of mathematical reasoning with transformers
Jordan Meadows, Marco Valentino, Damien Teney, and Andre Freitas · 2023
Later among the works it cites.
Tree of uncertain thoughts reasoning for large language models, 2023
Shentong Mo and Miao Xin · 2023
Later among the works it cites.
Why think step-by-step? reasoning emerges from the locality of experience
Ben Prystawski and Noah D Goodman · 2023
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Binding language models in symbolic languages
Zhoujun Cheng, Tianbao Xie, Peng Shi, Chengzu Li, Rahul Nadkarni, Yushi Hu, Caiming Xiong, Dragomir Radev, Mari Ostendorf, Luke Zettlemoyer, et al · 2023
Cited alongside, same era.
Faith and fate: Limits of transformers on compositionality
Nouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li, Liwei Jian, Bill Yuchen Lin, Peter West, Chandra Bhagavatula, Ronan Le Bras, Jena D Hwang, et al · 2023
Cited alongside, same era.
From interpolation to extrapolation: Complete length generalization for arithmetic transformers
Shaoxiong Duan and Yining Shi · 2023
Cited alongside, same era.
Towards revealing the mystery behind chain of thought: a theoretical perspective
Guhao Feng, Yuntian Gu, Bohang Zhang, Haotian Ye, Di He, and Liwei Wang · 2023
Cited alongside, same era.
Chain-of-thought hub: A continuous effort to measure large language models’ reasoning performance
Yao Fu, Litu Ou, Mingyu Chen, Yuhao Wan, Hao Peng, and Tushar Khot · 2023
Cited alongside, same era.
Large language models are not abstract reasoners
Gaël Gendron, Qiming Bao, Michael Witbrock, and Gillian Dobbie · 2023
Cited alongside, same era.
Pal: Program-aided language models
Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig · 2023
Cited alongside, same era.
Later among the works it cites.
Certified reasoning with language models
Gabriel Poesia, Kanishk Gandhi, Eric Zelikman, and Noah D Goodman · 2023
Later among the works it cites.
Jingyuan Qi, Zhiyang Xu, Ying Shen, Minqian Liu, Di Jin, Qifan Wang, and Lifu Huang · 2023
Later among the works it cites.
Randomized positional encodings boost length generalization of transformers
Anian Ruoss, Grégoire Delétang, Tim Genewein, Jordi Grau-Moya, Róbert Csordás, Mehdi Bennani, Shane Legg, and Joel Veness · 2023
Later among the works it cites.
Understanding arithmetic reasoning in language models using causal mediation analysis
Alessandro Stolfo, Yonatan Belinkov, and Mrinmaya Sachan · 2023
Later among the works it cites.
A length-extrapolatable transformer
Yutao Sun, Li Dong, Barun Patra, Shuming Ma, Shaohan Huang, Alon Benhaim, Vishrav Chaudhary, Xia Song, and Furu Wei · 2023
Later among the works it cites.
Scone: Benchmarking negation reasoning in language models with fine-tuning and in-context learning
Jingyuan Selena She, Christopher Potts, Samuel R Bowman, and Atticus Geiger · 2023
Later among the works it cites.
Invalid logic, equivalent gains: The bizarreness of reasoning in language model prompting
Rylan Schaeffer, Kateryna Pistunova, Samar Khanna, Sarthak Consul, and Sanmi Koyejo · 2023
Later among the works it cites.
Towards benchmarking and improving the temporal reasoning capability of large language models
Qingyu Tan, Hwee Tou Ng, and Lidong Bing · 2023
Later among the works it cites.
Large language models are in-context semantic reasoners rather than symbolic reasoners
Xiaojuan Tang, Zilong Zheng, Jiaqi Li, Fanxu Meng, Song-Chun Zhu, Yitao Liang, and Muhan Zhang · 2023
Later among the works it cites.
Exploring equation as a better intermediate meaning representation for numerical reasoning
Dingzirui Wang, Longxu Dou, Wenbin Zhang, Junyu Zeng, and Wanxiang Che · 2023
Later among the works it cites.
Learning multi-step reasoning by solving arithmetic tasks
Tianduo Wang and Wei Lu · 2023
Later among the works it cites.
Making large language models better reasoners with alignment
Peiyi Wang, Lei Li, Liang Chen, Feifan Song, Binghuai Lin, Yunbo Cao, Tianyu Liu, and Zhifang Sui · 2023
Later among the works it cites.
Sub-task decomposition enables learning in sequence to sequence tasks
Noam Wies, Yoav Levine, and Amnon Shashua · 2023
Later among the works it cites.
Zhaofeng Wu, Linlu Qiu, Alexis Ross, Ekin Akyürek, Boyuan Chen, Bailin Wang, Najoung Kim, Jacob Andreas, and Yoon Kim · 2023
Later among the works it cites.
Boosting language models reasoning with chain-of-knowledge prompting
Jianing Wang, Qiushi Sun, Nuo Chen, Xiang Li, and Ming Gao · 2023
Later among the works it cites.
Fangzhi Xu, Qika Lin, Jiawei Han, Tianzhe Zhao, Jun Liu, and Erik Cambria · 2023
Later among the works it cites.
Rewoo: Decoupling reasoning from observations for efficient augmented language models
Binfeng Xu, Zhiyuan Peng, Bowen Lei, Subhabrata Mukherjee, Yuchen Liu, and Dongkuan Xu · 2023
Later among the works it cites.
Beyond chain-of-thought, effective graph-of-thought reasoning in large language models
Yao Yao, Zuchao Li, and Hai Zhao · 2023
Later among the works it cites.
Thinking like an expert: Multimodal hypergraph-of-thought (hot) reasoning to boost foundation modals
Fanglong Yao, Changyuan Tian, Jintao Liu, Zequn Zhang, Qing Liu, Li Jin, Shuchao Li, Xiaoyu Li, and Xian Sun · 2023
Later among the works it cites.
Tree of thoughts: Deliberate problem solving with large language models
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L Griffiths, Yuan Cao, and Karthik Narasimhan · 2023
Later among the works it cites.
What algorithms can transformers learn? a study in length generalization
Hattie Zhou, Arwen Bradley, Etai Littwin, Noam Razin, Omid Saremi, Josh Susskind, Samy Bengio, and Preetum Nakkiran · 2023
Later among the works it cites.
Cumulative reasoning with large language models
Yifan Zhang, Jingqin Yang, Yang Yuan, and Andrew Chi-Chih Yao · 2023
Later among the works it cites.