From System 1 to System 2: A Survey of Reasoning Large Language Models

M. Shoeybi, M. Patwary, R. Puri, P. LeGresley, J. Casper, and B. Catanzaro, “Megatron-lm: Training multi-billion parameter language models using model parallelism,” 2020. [Online]. Available: https://arxiv.org/abs/1909.08053

Original

1909

Earlier work this paper cites.

C. I. Lewis, C. H. Langford, and P. Lamprecht, Symbolic logic . Dover publications New York, 1959, vol. 170

1959

Earlier work this paper cites.

M. Minsky et al. , “A framework for representing knowledge,” 1974

1974

Earlier work this paper cites.

J. McCarthy, “History of LISP,” in History of programming languages , 1978, pp. 173–185

1978

Earlier work this paper cites.

J. S. B. Evans, “Heuristic and analytic processes in reasoning,” British Journal of Psychology , vol. 75, no. 4, pp. 451–468, 1984

1984

Earlier work this paper cites.

D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” nature , vol. 323, no. 6088, pp. 533–536, 1986

1986

Earlier work this paper cites.

R. G. Jeroslow, “Computation-oriented reductions of predicate to propositional logic,” Decision Support Systems , vol. 4, no. 2, pp. 183–197, 1988

1988

Earlier work this paper cites.

A. Colmerauer, “An introduction to Prolog III,” Communications of the ACM , vol. 33, no. 7, pp. 69–90, 1990

1990

Earlier work this paper cites.

C. J. Watkins and P. Dayan, “Q-learning,” Machine learning , vol. 8, pp. 279–292, 1992

1992

Earlier work this paper cites.

Y. LeCun, Y. Bengio et al. , “Convolutional networks for images, speech, and time series,” The handbook of brain theory and neural networks , vol. 3361, no. 10, p. 1995, 1995

1995

Earlier work this paper cites.

S. Hochreiter, “Long Short-term Memory,” Neural Computation MIT-Press , 1997

1997

Earlier work this paper cites.

K. R. Apt et al. , From logic programming to Prolog . Prentice Hall London, 1997, vol. 362

1997

Earlier work this paper cites.

R. S. Sutton, A. G. Barto et al. , Reinforcement learning: An introduction . MIT press Cambridge, 1998, vol. 1, no. 1

1998

Earlier work this paper cites.

M. P. Singh, A. S. Rao, and M. P. Georgeff, Formal methods in DAI: Logic-based representation and reasoning . MIT Press Cambridge, 1999, vol. 8

1999

Earlier work this paper cites.

A. Y. Ng, D. Harada, and S. Russell, “Policy invariance under reward transformations: Theory and application to reward shaping,” in Icml , vol. 99. Citeseer, 1999, pp. 278–287

1999

Earlier work this paper cites.

L. Bachmair and H. Ganzinger, “Resolution Theorem Proving.” Handbook of automated reasoning , vol. 1, no. 02, 2001

2001

Earlier work this paper cites.

D. Kahneman, “Maps of bounded rationality: Psychology for behavioral economics,” American economic review , vol. 93, no. 5, pp. 1449–1475, 2003

2003

Earlier work this paper cites.

W. F. Clocksin and C. S. Mellish, Programming in PROLOG . Springer Science & Business Media, 2003

2003

Earlier work this paper cites.

G. E. Hinton, S. Osindero, and Y.-W. Teh, “A fast learning algorithm for deep belief nets,” Neural computation , vol. 18, no. 7, pp. 1527–1554, 2006

2006

Earlier work this paper cites.

G. E. Hinton and R. R. Salakhutdinov, “Reducing the dimensionality of data with neural networks,” science , vol. 313, no. 5786, pp. 504–507, 2006

2006

Earlier work this paper cites.

S. Gelly and D. Silver, “Monte-Carlo tree search and rapid action value estimation in computer Go,” Artificial Intelligence , vol. 175, no. 11, pp. 1856–1875, 2011

2011

Earlier work this paper cites.

G. Hinton, L. Deng, D. Yu, G. E. Dahl, A.-r. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath et al. , “Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups,” IEEE Signal processing magazine , vol. 29, no. 6, pp. 82–97, 2012

2012

Earlier work this paper cites.

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Advances in neural information processing systems , vol. 25, 2012

2012

Earlier work this paper cites.

R. Carnap, Introduction to symbolic logic and its applications . Courier Corporation, 2012

2012

Earlier work this paper cites.

C. B. Browne, E. Powley, D. Whitehouse, S. M. Lucas, P. I. Cowling, P. Rohlfshagen, S. Tavener, D. Perez, S. Samothrakis, and S. Colton, “A survey of monte carlo tree search methods,” IEEE Transactions on Computational Intelligence and AI in games , vol. 4, no. 1, pp. 1–43, 2012

2012

Earlier work this paper cites.

K. Cho, B. van Merrienboer, Ç. Gülçehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, “Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation,” in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL , 2014, pp. 1724–1734

2014

Earlier work this paper cites.

I. Sutskever, “Sequence to Sequence Learning with Neural Networks,” arXiv preprint arXiv:1409.3215 , 2014

Original

2014

Earlier work this paper cites.

N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: a simple way to prevent neural networks from overfitting,” The journal of machine learning research , vol. 15, no. 1, pp. 1929–1958, 2014

2014

Earlier work this paper cites.

D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 , 2014

Original

2014

Earlier work this paper cites.

Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” nature , vol. 521, no. 7553, pp. 436–444, 2015

2015

Earlier work this paper cites.

V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski et al. , “Human-level control through deep reinforcement learning,” nature , vol. 518, no. 7540, pp. 529–533, 2015

2015

Earlier work this paper cites.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778

2016

Earlier work this paper cites.

D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot et al. , “Mastering the game of Go with deep neural networks and tree search,” nature , vol. 529, no. 7587, pp. 484–489, 2016

2016

Earlier work this paper cites.

A. Vaswani, “Attention is all you need,” Advances in Neural Information Processing Systems , 2017

2017

Earlier work this paper cites.

G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 4700–4708

2017

Earlier work this paper cites.

D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, T. Hubert, L. Baker, M. Lai, A. Bolton et al. , “Mastering the game of go without human knowledge,” nature , vol. 550, no. 7676, pp. 354–359, 2017

2017

Earlier work this paper cites.

T. Anthony, Z. Tian, and D. Barber, “Thinking Fast and Slow with Deep Learning and Tree Search,” 2017. [Online]. Available: https://arxiv.org/abs/1705.08439

Original

2017

Earlier work this paper cites.

A. Radford, “Improving language understanding by generative pre-training,” 2018

2018

Earlier work this paper cites.

R. R. Torrado, P. Bontrager, J. Togelius, J. Liu, and D. Perez-Liebana, “Deep reinforcement learning for general video game ai,” in 2018 IEEE Conference on Computational Intelligence and Games (CIG) . IEEE, 2018, pp. 1–8

2018

Earlier work this paper cites.

J. Devlin, M. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers) , 2019, pp. 4171–4186

2019

Earlier work this paper cites.

Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, “RoBERTa: A Robustly Optimized BERT Pretraining Approach,” CoRR , vol. abs/1907.11692, 2019

Original

2019

Earlier work this paper cites.

A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al. , “Language models are unsupervised multitask learners,” OpenAI blog , vol. 1, no. 8, p. 9, 2019

2019

Earlier work this paper cites.

O. Vinyals, I. Babuschkin, W. M. Czarnecki, M. Mathieu, A. Dudzik, J. Chung, D. H. Choi, R. Powell, T. Ewalds, P. Georgiev et al. , “Grandmaster level in StarCraft II using multi-agent reinforcement learning,” nature , vol. 575, no. 7782, pp. 350–354, 2019

2019

Earlier work this paper cites.

F. Chollet, “On the measure of intelligence,” arXiv preprint arXiv:1911.01547 , 2019

Original

2019

Earlier work this paper cites.

T. Nandy, M. Y. I. B. Idris, R. M. Noor, L. M. Kiah, L. S. Lun, N. B. A. Juma’at, I. Ahmedy, N. A. Ghani, and S. Bhattacharyya, “Review on security of internet of things authentication mechanism,” IEEE Access , vol. 7, pp. 151 054–151 089, 2019

2019

Earlier work this paper cites.

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al. , “Language models are few-shot learners,” Advances in neural information processing systems , vol. 33, pp. 1877–1901, 2020

2020

Earlier work this paper cites.

I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial networks,” Communications of the ACM , vol. 63, no. 11, pp. 139–144, 2020

2020

Earlier work this paper cites.

K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano et al. , “Training verifiers to solve math word problems,” arXiv preprint arXiv:2110.14168 , 2021

Original

2021

Earlier work this paper cites.

A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al. , “Learning transferable visual models from natural language supervision,” in International conference on machine learning . PMLR, 2021, pp. 8748–8763

2021

Earlier work this paper cites.

A. Ramesh, M. Pavlov, G. Goh, S. Gray, C. Voss, A. Radford, M. Chen, and I. Sutskever, “Zero-shot text-to-image generation,” in International conference on machine learning . Pmlr, 2021, pp. 8821–8831

2021

Earlier work this paper cites.

J. He, J. Qiu, A. Zeng, Z. Yang, J. Zhai, and J. Tang, “Fastmoe: A fast mixture-of-expert training system,” arXiv preprint arXiv:2103.13262 , 2021

Original

2021

Earlier work this paper cites.

B. Lester, R. Al-Rfou, and N. Constant, “The Power of Scale for Parameter-Efficient Prompt Tuning,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , 2021, pp. 3045–3059

2021

Earlier work this paper cites.

D. Jin, E. Pan, N. Oufattole, W.-H. Weng, H. Fang, and P. Szolovits, “What disease does this patient have? a large-scale open domain question answering dataset from medical exams,” Applied Sciences , vol. 11, no. 14, p. 6421, 2021

2021

Earlier work this paper cites.

W. Hua and Y. Zhang, “System 1+ system 2= better world: Neural-symbolic chain of logic reasoning,” in Findings of the Association for Computational Linguistics: EMNLP 2022 , 2022, pp. 601–612

2022

Earlier work this paper cites.

J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou et al. , “Chain-of-thought prompting elicits reasoning in large language models,” Advances in neural information processing systems , vol. 35, pp. 24 824–24 837, 2022

2022

Earlier work this paper cites.

L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al. , “Training language models to follow instructions with human feedback,” Advances in neural information processing systems , vol. 35, pp. 27 730–27 744, 2022

2022

Earlier work this paper cites.

T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, “Large language models are zero-shot reasoners,” Advances in neural information processing systems , vol. 35, pp. 22 199–22 213, 2022

2022

Earlier work this paper cites.

J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds et al. , “Flamingo: a visual language model for few-shot learning,” Advances in Neural Information Processing Systems , vol. 35, pp. 23 716–23 736, 2022

2022

Earlier work this paper cites.

N. Du, Y. Huang, A. M. Dai, S. Tong, D. Lepikhin, Y. Xu, M. Krikun, Y. Zhou, A. W. Yu, O. Firat et al. , “Glam: Efficient scaling of language models with mixture-of-experts,” in International conference on machine learning . PMLR, 2022, pp. 5547–5569

2022

Earlier work this paper cites.

S. Min, X. Lyu, A. Holtzman, M. Artetxe, M. Lewis, H. Hajishirzi, and L. Zettlemoyer, “Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?” in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , 2022, pp. 11 048–11 064

2022

Earlier work this paper cites.

J. Uesato, N. Kushman, R. Kumar, F. Song, N. Siegel, L. Wang, A. Creswell, G. Irving, and I. Higgins, “Solving math word problems with process-and outcome-based feedback,” arXiv preprint arXiv:2211.14275 , 2022

Original

2022

Earlier work this paper cites.

Y. Li, Z. Lin, S. Zhang, Q. Fu, B. Chen, J.-G. Lou, and W. Chen, “Making large language models better reasoners with step-aware verifier,” arXiv preprint arXiv:2206.02336 , 2022

Original

2022

Earlier work this paper cites.

J. Uesato, N. Kushman, R. Kumar, F. Song, N. Siegel, L. Wang, A. Creswell, G. Irving, and I. Higgins, “Solving math word problems with process- and outcome-based feedback,” 2022. [Online]. Available: https://arxiv.org/abs/2211.14275

Original

2022

Earlier work this paper cites.

E. Zelikman, Y. Wu, J. Mu, and N. D. Goodman, “STaR: Bootstrapping Reasoning With Reasoning,” in Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022 , 2022

2022

Earlier work this paper cites.

W. Chen, X. Ma, X. Wang, and W. W. Cohen, “Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks,” arXiv preprint arXiv:2211.12588 , 2022

Original

2022

Earlier work this paper cites.

T. Khot, H. Trivedi, M. Finlayson, Y. Fu, K. Richardson, P. Clark, and A. Sabharwal, “Decomposed prompting: A modular approach for solving complex tasks,” arXiv preprint arXiv:2210.02406 , 2022

Original

2022

Earlier work this paper cites.

D. Zhou, N. Schärli, L. Hou, J. Wei, N. Scales, X. Wang, D. Schuurmans, C. Cui, O. Bousquet, Q. Le et al. , “Least-to-most prompting enables complex reasoning in large language models,” arXiv preprint arXiv:2205.10625 , 2022

Original

2022

Earlier work this paper cites.

M. Suzgun, N. Scales, N. Schärli, S. Gehrmann, Y. Tay, H. W. Chung, A. Chowdhery, Q. V. Le, E. H. Chi, D. Zhou et al. , “Challenging big-bench tasks and whether chain-of-thought can solve them,” arXiv preprint arXiv:2210.09261 , 2022

Original

2022

Earlier work this paper cites.

Y. Li, D. Choi, J. Chung, N. Kushman, J. Schrittwieser, R. Leblond, T. Eccles, J. Keeling, F. Gimeno, A. Dal Lago et al. , “Competition-level code generation with alphacode,” Science , vol. 378, no. 6624, pp. 1092–1097, 2022

2022

Earlier work this paper cites.

S. Yao, H. Chen, J. Yang, and K. Narasimhan, “Webshop: Towards scalable real-world web interaction with grounded language agents,” Advances in Neural Information Processing Systems , vol. 35, pp. 20 744–20 757, 2022

2022

Earlier work this paper cites.

R. Wang, P. Jansen, M.-A. Côté, and P. Ammanabrolu, “ScienceWorld: Is your Agent Smarter than a 5th Grader?” 2022. [Online]. Available: https://arxiv.org/abs/2203.07540

Original

2022

Earlier work this paper cites.

P. Lu, S. Mishra, T. Xia, L. Qiu, K.-W. Chang, S.-C. Zhu, O. Tafjord, P. Clark, and A. Kalyan, “Learn to explain: Multimodal reasoning via thought chains for science question answering,” Advances in Neural Information Processing Systems , vol. 35, pp. 2507–2521, 2022

2022

Earlier work this paper cites.

X. Wang, J. Wei, D. Schuurmans, Q. V. Le, E. H. Chi, S. Narang, A. Chowdhery, and D. Zhou, “Self-Consistency Improves Chain of Thought Reasoning in Language Models,” in The Eleventh International Conference on Learning Representations , 2023

2023

Earlier work this paper cites.

D. Zhou, N. Schärli, L. Hou, J. Wei, N. Scales, X. Wang, D. Schuurmans, C. Cui, O. Bousquet, Q. V. Le et al. , “Least-to-Most Prompting Enables Complex Reasoning in Large Language Models,” in The Eleventh International Conference on Learning Representations , 2023

2023

Earlier work this paper cites.

J. Huang and K. C.-C. Chang, “Towards Reasoning in Large Language Models: A Survey,” in Findings of the Association for Computational Linguistics: ACL 2023 , 2023, pp. 1049–1065

2023

Earlier work this paper cites.

S. Qiao, Y. Ou, N. Zhang, X. Chen, Y. Yao, S. Deng, C. Tan, F. Huang, and H. Chen, “Reasoning with Language Model Prompting: A Survey,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2023, pp. 5368–5393

2023

Earlier work this paper cites.

B. Wang, S. Min, X. Deng, J. Shen, Y. Wu, L. Zettlemoyer, and H. Sun, “Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2023, pp. 2717–2739

2023

Earlier work this paper cites.

O. Shaikh, H. Zhang, W. Held, M. Bernstein, and D. Yang, “On Second Thought, Let’s Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2023, pp. 4454–4470

2023

Earlier work this paper cites.

Z. Zhang, A. Zhang, M. Li, and A. Smola, “Automatic Chain of Thought Prompting in Large Language Models,” in The Eleventh International Conference on Learning Representations , 2023

2023

Earlier work this paper cites.

S. Hao, Y. Gu, H. Ma, J. Hong, Z. Wang, D. Wang, and Z. Hu, “Reasoning with Language Model is Planning with World Model,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , 2023, pp. 8154–8173

2023

Earlier work this paper cites.

Y. Zhang, “Meta prompting for agi systems,” arXiv preprint arXiv:2311.11482 , 2023

Original

2023

Earlier work this paper cites.

H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar et al. , “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971 , 2023

Original

2023

Earlier work this paper cites.

W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y. Hou, Y. Min, B. Zhang, J. Zhang, Z. Dong et al. , “A survey of large language models,” arXiv preprint arXiv:2303.18223 , 2023

Original

2023

Earlier work this paper cites.

H. Liu, C. Li, Q. Wu, and Y. J. Lee, “Visual Instruction Tuning,” in Thirty-seventh Conference on Neural Information Processing Systems , 2023

2023

Earlier work this paper cites.

Y. Liu, A. Singh, C. D. Freeman, J. D. Co-Reyes, and P. J. Liu, “Improving large language model fine-tuning for solving math problems,” arXiv preprint arXiv:2310.10047 , 2023

Original

2023

Earlier work this paper cites.

X. Zhu, J. Wang, L. Zhang, Y. Zhang, Y. Huang, R. Gan, J. Zhang, and Y. Yang, “Solving Math Word Problems via Cooperative Reasoning induced Language Models,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2023, pp. 4471–4485

2023

Earlier work this paper cites.

P. Lu, L. Qiu, K.-W. Chang, Y. N. Wu, S.-C. Zhu, T. Rajpurohit, P. Clark, and A. Kalyan, “Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning,” in The Eleventh International Conference on Learning Representations , 2023

2023

Earlier work this paper cites.

F. Yao, C. Tian, J. Liu, Z. Zhang, Q. Liu, L. Jin, S. Li, X. Li, and X. Sun, “Thinking like an expert: Multimodal hypergraph-of-thought (hot) reasoning to boost foundation modals,” arXiv preprint arXiv:2308.06207 , 2023

Original

2023

Earlier work this paper cites.

Y. Yao, Z. Li, and H. Zhao, “Beyond Chain-of-Thought, Effective Graph-of-Thought Reasoning in Language Models,” arXiv preprint arXiv:2305.16582 , 2023

Original

2023

Earlier work this paper cites.

Y. Wen, Z. Wang, and J. Sun, “Mindmap: Knowledge graph prompting sparks graph of thoughts in large language models,” arXiv preprint arXiv:2308.09729 , 2023

Original

2023

Earlier work this paper cites.

B. Lei, C. Liao, C. Ding et al. , “Boosting logical reasoning in large language models through a new framework: The graph of thought,” arXiv preprint arXiv:2308.08614 , 2023

Original

2023

Earlier work this paper cites.

H. You, R. Sun, Z. Wang, L. Chen, G. Wang, H. Ayyubi, K.-W. Chang, and S.-F. Chang, “IdealGPT: Iteratively Decomposing Vision and Language Reasoning via Large Language Models,” in Findings of the Association for Computational Linguistics: EMNLP 2023 , 2023, pp. 11 289–11 303

2023

Earlier work this paper cites.

OpenAI, “GPT-4 Technical Report,” 2023

2023

Earlier work this paper cites.

J. Li, D. Li, S. Savarese, and S. C. H. Hoi, “BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models,” in International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA , 2023, pp. 19 730–19 742

2023

Earlier work this paper cites.

W. Dai, J. Li, D. Li, A. M. H. Tiong, J. Zhao, W. Wang, B. Li, P. Fung, and S. C. H. Hoi, “InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning,” in Thirty-seventh Conference on Neural Information Processing Systems , 2023

2023

Earlier work this paper cites.

J. White, Q. Fu, S. Hays, M. Sandborn, C. Olea, H. Gilbert, A. Elnashar, J. Spencer-Smith, and D. C. Schmidt, “A prompt pattern catalog to enhance prompt engineering with chatgpt,” arXiv preprint arXiv:2302.11382 , 2023

Original

2023

Earlier work this paper cites.

M. Świechowski, K. Godlewski, B. Sawicki, and J. Mańdziuk, “Monte Carlo tree search: A review of recent modifications and applications,” Artificial Intelligence Review , vol. 56, no. 3, pp. 2497–2562, 2023

2023

Earlier work this paper cites.

H. Luo, Q. Sun, C. Xu, P. Zhao, J. Lou, C. Tao, X. Geng, Q. Lin, S. Chen, and D. Zhang, “Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct,” arXiv preprint arXiv:2308.09583 , 2023

Original

2023

Earlier work this paper cites.

Z. Zhao, W. S. Lee, and D. Hsu, “Large Language Models as Commonsense Knowledge for Large-Scale Task Planning,” in Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023 , 2023

2023

Earlier work this paper cites.

H. Xu, “No Train Still Gain. Unleash Mathematical Reasoning of Large Language Models with Monte Carlo Tree Search Guided by Energy Function,” CoRR , vol. abs/2309.03224, 2023

Original

2023

Earlier work this paper cites.

H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe, “Let’s verify step by step,” in The Twelfth International Conference on Learning Representations , 2023

2023

Earlier work this paper cites.

F. Yu, A. Gao, and B. Wang, “Outcome-supervised verifiers for planning in mathematical reasoning,” arXiv preprint arXiv:2311.09724 , 2023

Original

2023

Earlier work this paper cites.

Y. Li, Z. Lin, S. Zhang, Q. Fu, B. Chen, J.-G. Lou, and W. Chen, “Making language models better reasoners with step-aware verifier,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2023, pp. 5315–5333

2023

Earlier work this paper cites.

Z. Wu, Y. Hu, W. Shi, N. Dziri, A. Suhr, P. Ammanabrolu, N. A. Smith, M. Ostendorf, and H. Hajishirzi, “Fine-grained human feedback gives better rewards for language model training,” Advances in Neural Information Processing Systems , vol. 36, pp. 59 008–59 033, 2023

2023

Earlier work this paper cites.

Ç. Gülçehre, T. L. Paine, S. Srinivasan, K. Konyushkova, L. Weerts, A. Sharma, A. Siddhant, A. Ahern, M. Wang, C. Gu, W. Macherey, A. Doucet, O. Firat, and N. de Freitas, “Reinforced Self-Training (ReST) for Language Modeling,” CoRR , vol. abs/2308.08998, 2023

Original

2023

Earlier work this paper cites.

Y. Dubois, C. X. Li, R. Taori, T. Zhang, I. Gulrajani, J. Ba, C. Guestrin, P. Liang, and T. B. Hashimoto, “AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback,” in Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023 , 2023

2023

Earlier work this paper cites.

A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y. Yang, S. Gupta, B. P. Majumder, K. Hermann, S. Welleck, A. Yazdanbakhsh, and P. Clark, “Self-Refine: Iterative Refinement with Self-Feedback,” in Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023 , 2023

2023

Earlier work this paper cites.

Y. Weng, M. Zhu, F. Xia, B. Li, S. He, S. Liu, B. Sun, K. Liu, and J. Zhao, “Large Language Models are Better Reasoners with Self-Verification,” in Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023 . Association for Computational Linguistics, 2023, pp. 2550–2575

2023

Earlier work this paper cites.

Y. Xie, K. Kawaguchi, Y. Zhao, X. Zhao, M.-Y. Kan, J. He, and Q. Xie, “Self-Evaluation Guided Beam Search for Reasoning,” 2023. [Online]. Available: https://arxiv.org/abs/2305.00633

Original

2023

Earlier work this paper cites.

Y. Yao, H. Wu, Q. Xu, and L. Song, “Fine-grained Conversational Decoding via Isotropic and Proximal Search,” 2023. [Online]. Available: https://arxiv.org/abs/2310.08130

Original

2023

Earlier work this paper cites.

N. Xu, C. Zhou, A. Celikyilmaz, and X. Ma, “Look-back Decoding for Open-Ended Text Generation,” 2023. [Online]. Available: https://arxiv.org/abs/2305.13477

Original

2023

Earlier work this paper cites.

N. Miao, Y. W. Teh, and T. Rainforth, “Selfcheck: Using llms to zero-shot check their own step-by-step reasoning,” arXiv preprint arXiv:2308.00436 , 2023

Original

2023

Earlier work this paper cites.

S. An, Z. Ma, Z. Lin, N. Zheng, J.-G. Lou, and W. Chen, “Learning from mistakes makes llm better reasoner,” arXiv preprint arXiv:2310.20689 , 2023

Original

2023

Earlier work this paper cites.

L. Gao, A. Madaan, S. Zhou, U. Alon, P. Liu, Y. Yang, J. Callan, and G. Neubig, “Pal: Program-aided language models,” in International Conference on Machine Learning . PMLR, 2023, pp. 10 764–10 799

2023

Earlier work this paper cites.

H. Lee, S. Phatale, H. Mansoor, K. R. Lu, T. Mesnard, J. Ferret, C. Bishop, E. Hall, V. Carbune, and A. Rastogi, “Rlaif: Scaling reinforcement learning from human feedback with ai feedback,” 2023

2023

Earlier work this paper cites.

S. Zhou, F. F. Xu, H. Zhu, X. Zhou, R. Lo, A. Sridhar, X. Cheng, Y. Bisk, D. Fried, U. Alon et al. , “WebArena: A Realistic Web Environment for Building Autonomous Agents,” arXiv preprint arXiv:2307.13854 , 2023. [Online]. Available: https://webarena.dev

Original

2023

Earlier work this paper cites.

M.-L. Zhang, F. Yin, and C.-L. Liu, “A Multi-Modal Neural Geometric Solver with Textual Clauses Parsed from Diagram,” in IJCAI , 2023

2023

Earlier work this paper cites.

J. Yu, R. He, and R. Ying, “Thought propagation: An analogical approach to complex reasoning with large language models,” arXiv preprint arXiv:2310.03965 , 2023

Original

2023

Earlier work this paper cites.

Z.-Z. Li, M.-L. Zhang, F. Yin, and C.-L. Liu, “Lans: A layout-aware neural solver for plane geometry problem,” arXiv preprint arXiv:2311.16476 , 2023

Original

2023

Earlier work this paper cites.

O. Contributors, “OpenCompass: A Universal Evaluation Platform for Foundation Models,” 2023. [Online]. Available: https://github.com/open-compass/opencompass

2023

Earlier work this paper cites.

W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica, “Efficient memory management for large language model serving with pagedattention,” 2023. [Online]. Available: https://arxiv.org/abs/2309.06180

Original

2023

Earlier work this paper cites.

H. Liu, S. Dass, R. Martín-Martín, and Y. Zhu, “Model-based runtime monitoring with interactive imitation learning,” 2023. [Online]. Available: https://arxiv.org/abs/2310.17552

Original

2023

Earlier work this paper cites.

Y.-S. Chuang, Y. Xie, H. Luo, Y. Kim, J. Glass, and P. He, “Dola: Decoding by contrasting layers improves factuality in large language models,” arXiv preprint arXiv:2309.03883 , 2023

Original

2023

Earlier work this paper cites.

A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” arXiv preprint arXiv:2312.00752 , 2023

Original

2023

Earlier work this paper cites.

B. Peng, E. Alcaide, Q. Anthony, A. Albalak, S. Arcadinho, S. Biderman, H. Cao, X. Cheng, M. Chung, M. Grella, K. K. GV, X. He, H. Hou, J. Lin, P. Kazienko, J. Kocon, J. Kong, B. Koptyra, H. Lau, K. S. I. Mantri, F. Mom, A. Saito, G. Song, X. Tang, B. Wang, J. S. Wind, S. Wozniak, R. Zhang, Z. Zhang, Q. Zhao, P. Zhou, Q. Zhou, J. Zhu, and R.-J. Zhu, “Rwkv: Reinventing rnns for the transformer era,” 2023. [Online]. Available: https://arxiv.org/abs/2305.13048

Original

2023

Earlier work this paper cites.

Y. Leviathan, M. Kalman, and Y. Matias, “Fast inference from transformers via speculative decoding,” in International Conference on Machine Learning , 2023, pp. 19 274–19 286

2023

Earlier work this paper cites.

X. Ning, Z. Lin, Z. Zhou, Z. Wang, H. Yang, and Y. Wang, “Skeleton-of-thought: Large language models can do parallel decoding,” Proceedings ENLSP-III , 2023

2023

Earlier work this paper cites.

X. Miao, G. Oliaro, Z. Zhang, X. Cheng, Z. Wang, Z. Zhang, R. Y. Y. Wong, A. Zhu, L. Yang, X. Shi et al. , “SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification,” arXiv preprint arXiv:2305.09781 , 2023

Original

2023

Earlier work this paper cites.

P. Hu, J. Qi, X. Li, H. Li, X. Wang, B. Quan, R. Wang, and Y. Zhou, “Tree-of-mixed-thought: Combining fast and slow thinking for multi-hop visual reasoning,” arXiv preprint arXiv:2308.09658 , 2023

Original

2023

Earlier work this paper cites.

G. Team, R. Anil, S. Borgeaud, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, K. Millican et al. , “Gemini: a family of highly capable multimodal models,” arXiv preprint arXiv:2312.11805 , 2023

Original

2023

Earlier work this paper cites.

N. Chen, Z. Zheng, N. Wu, L. Shou, M. Gong, Y. Song, D. Zhang, and J. Li, “Breaking language barriers in multilingual mathematical reasoning: Insights and observations,” arXiv preprint arXiv:2310.20246 , 2023

Original

2023

Earlier work this paper cites.

E. Zelikman, Y. Wu, J. Mu, and N. D. Goodman, “STaR: Self-taught reasoner bootstrapping reasoning with reasoning,” in Proc. the 36th International Conference on Neural Information Processing Systems , vol. 1126, 2024

2024

Earlier work this paper cites.

H. Shao, S. Qian, H. Xiao, G. Song, Z. Zong, L. Wang, Y. Liu, and H. Li, “Visual cot: Advancing multi-modal language models with a comprehensive dataset and benchmark for chain-of-thought reasoning,” in The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track , 2024

2024

Earlier work this paper cites.

OpenAI, “Hello GPT-4o,” May 2024. [Online]. Available: https://openai.com/index/hello-gpt-4o/

2024

Earlier work this paper cites.

A. Liu, B. Feng, B. Xue, B. Wang, B. Wu, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan et al. , “Deepseek-v3 technical report,” arXiv preprint arXiv:2412.19437 , 2024

Original

2024

Earlier work this paper cites.

D. Zhang, Y. Yu, J. Dong, C. Li, D. Su, C. Chu, and D. Yu, “MM-LLMs: Recent Advances in MultiModal Large Language Models,” in Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024 . Association for Computational Linguistics, 2024, pp. 12 401–12 430

2024

Earlier work this paper cites.

OpenAI, “Learning to reason with LLMs,” September 2024. [Online]. Available: https://openai.com/index/learning-to-reason-with-llms/

2024

Earlier work this paper cites.

H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe, “Let’s Verify Step by Step,” in The Twelfth International Conference on Learning Representations , 2024

2024

Earlier work this paper cites.

M. Jin, Q. Yu, D. Shu, H. Zhao, W. Hua, Y. Meng, Y. Zhang, and M. Du, “The impact of reasoning step length on large language models,” arXiv preprint arXiv:2401.04925 , 2024

Original

2024

Earlier work this paper cites.

M. Besta, N. Blach, A. Kubicek, R. Gerstenberger, M. Podstawski, L. Gianinazzi, J. Gajda, T. Lehmann, H. Niewiadomski, P. Nyczyk et al. , “Graph of thoughts: Solving elaborate problems with large language models,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 16, 2024, pp. 17 682–17 690

2024

Earlier work this paper cites.

P. Cheng, T. Hu, H. Xu, Z. Zhang, Y. Dai, L. Han, and N. Du, “Self-playing Adversarial Language Game Enhances LLM Reasoning,” arXiv preprint arXiv:2404.10642 , 2024

Original

2024

Earlier work this paper cites.

P. Wu and S. Xie, “V?: Guided Visual Search as a Core Mechanism in Multimodal LLMs,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 13 084–13 094

2024

Earlier work this paper cites.

Z. Chen, R. Sun, W. Liu, Y. Hong, and C. Gan, “GENOME: Generative Neuro-Symbolic Visual Reasoning by Growing and Reusing Modules,” in International Conference on Learning Representations , 2024

2024

Earlier work this paper cites.

S. Wu, Z. Peng, X. Du, T. Zheng, M. Liu, J. Wu, J. Ma, Y. Li, J. Yang, W. Zhou et al. , “A Comparative Study on Reasoning Patterns of OpenAI’s o1 Model,” arXiv preprint arXiv:2410.13639 , 2024

Original

2024

Earlier work this paper cites.

Y. Qin, X. Li, H. Zou, Y. Liu, S. Xia, Z. Huang, Y. Ye, W. Yuan, H. Liu, Y. Li et al. , “O1 Replication Journey: A Strategic Progress Report–Part 1,” arXiv preprint arXiv:2410.18982 , 2024

Original

2024

Earlier work this paper cites.

Z. Huang, H. Zou, X. Li, Y. Liu, Y. Zheng, E. Chern, S. Xia, Y. Qin, W. Yuan, and P. Liu, “O1 Replication Journey–Part 2: Surpassing O1-preview through Simple Distillation, Big Progress or Bitter Lesson?” arXiv preprint arXiv:2411.16489 , 2024

Original

2024

Earlier work this paper cites.

Y. Min, Z. Chen, J. Jiang, J. Chen, J. Deng, Y. Hu, Y. Tang, J. Wang, X. Cheng, H. Song, W. X. Zhao, Z. Liu, Z. Wang, and J.-R. Wen, “Imitate, Explore, and Self-Improve: A Reproduction Report on Slow-thinking Reasoning Systems,” arXiv preprint arXiv:2412.09413 , 2024

Original

2024

Earlier work this paper cites.

Z. Zeng, Q. Cheng, Z. Yin, B. Wang, S. Li, Y. Zhou, Q. Guo, X. Huang, and X. Qiu, “Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspective,” arXiv preprint arXiv:2412.14135 , 2024

Original

2024

Earlier work this paper cites.

Y. Zhang, S. Mao, T. Ge, X. Wang, A. de Wynter, Y. Xia, W. Wu, T. Song, M. Lan, and F. Wei, “LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Models,” arXiv preprint arXiv:2404.01230 , 2024

Original

2024

Earlier work this paper cites.

D. Dai, C. Deng, C. Zhao, R. Xu, H. Gao, D. Chen, J. Li, W. Zeng, X. Yu, Y. Wu et al. , “DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2024, pp. 1280–1297

2024

Earlier work this paper cites.

Q. Dong, L. Li, D. Dai, C. Zheng, J. Ma, R. Li, H. Xia, J. Xu, Z. Wu, B. Chang et al. , “A survey on in-context learning,” in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , 2024, pp. 1107–1128

2024

Earlier work this paper cites.

Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. Li, Y. Wu et al. , “Deepseekmath: Pushing the limits of mathematical reasoning in open language models,” arXiv preprint arXiv:2402.03300 , 2024

Original

2024

Earlier work this paper cites.

E. Zelikman, G. Harik, Y. Shao, V. Jayasiri, N. Haber, and N. D. Goodman, “Quiet-star: Language models can teach themselves to think before speaking,” arXiv preprint arXiv:2403.09629 , 2024

Original

2024

Earlier work this paper cites.

Y. Zhao, H. Yin, B. Zeng, H. Wang, T. Shi, C. Lyu, L. Wang, W. Luo, and K. Zhang, “Marco-o1: Towards open reasoning models for open-ended solutions,” arXiv preprint arXiv:2411.14405 , 2024

Original

2024

Earlier work this paper cites.

J. Chen, Z. Cai, K. Ji, X. Wang, W. Liu, R. Wang, J. Hou, and B. Wang, “Huatuogpt-o1, towards medical complex reasoning with llms,” arXiv preprint arXiv:2412.18925 , 2024

Original

2024

Earlier work this paper cites.

G. Sun, M. Jin, Z. Wang, C.-L. Wang, S. Ma, Q. Wang, Y. N. Wu, Y. Zhang, and D. Liu, “Visual agents as fast and slow thinkers,” arXiv preprint arXiv:2408.08862 , 2024

Original

2024

Earlier work this paper cites.

H. Wei, Y. Yin, Y. Li, J. Wang, L. Zhao, J. Sun, Z. Ge, and X. Zhang, “Slow Perception: Let’s Perceive Geometric Figures Step-by-step,” arXiv preprint arXiv:2412.20631 , 2024

Original

2024

Earlier work this paper cites.

Q. Team, “QwQ: Reflect Deeply on the Boundaries of the Unknown,” November 2024. [Online]. Available: https://qwenlm.github.io/blog/qwq-32b-preview/

2024

Earlier work this paper cites.

Y. Zhang, S. Wu, Y. Yang, J. Shu, J. Xiao, C. Kong, and J. Sang, “o1-coder: an o1 replication for coding,” arXiv preprint arXiv:2412.00154 , 2024

Original

2024

Earlier work this paper cites.

J. Wang, F. Meng, Y. Liang, and J. Zhou, “DRT-o1: Optimized Deep Reasoning Translation via Long Chain-of-Thought,” arXiv preprint arXiv:2412.17498 , 2024

Original

2024

Earlier work this paper cites.

X. Chen, J. Xu, T. Liang, Z. He, J. Pang, D. Yu, L. Song, Q. Liu, M. Zhou, Z. Zhang et al. , “Do NOT Think That Much for 2 + 3=? On the Overthinking of o1-Like LLMs,” arXiv preprint arXiv:2412.21187 , 2024

Original

2024

Earlier work this paper cites.

Z. Wan, X. Feng, M. Wen, S. M. McAleer, Y. Wen, W. Zhang, and J. Wang, “Alphazero-like tree-search can guide large language model decoding and training,” in Forty-first International Conference on Machine Learning , 2024

2024

Earlier work this paper cites.

Z. Bi, K. Han, C. Liu, Y. Tang, and Y. Wang, “Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning,” CoRR , vol. abs/2412.09078, 2024

Original

2024

Earlier work this paper cites.

J. Li, H. Le, Y. Zhou, C. Xiong, S. Savarese, and D. Sahoo, “CodeTree: Agent-guided Tree Search for Code Generation with Large Language Models,” CoRR , vol. abs/2411.04329, 2024

Original

2024

Earlier work this paper cites.

J. Qiu, Y. Lu, Y. Zeng, J. Guo, J. Geng, H. Wang, K. Huang, Y. Wu, and M. Wang, “TreeBoN: Enhancing Inference-Time Alignment with Speculative Tree-Search and Best-of-N Sampling,” CoRR , vol. abs/2410.16033, 2024

Original

2024

Earlier work this paper cites.

N. Dainese, M. Merler, M. Alakuijala, and P. Marttinen, “Generating Code World Models with Large Language Models Guided by Monte Carlo Tree Search,” CoRR , vol. abs/2405.15383, 2024

Original

2024

Earlier work this paper cites.

Q. Li, W. Xia, K. Du, X. Dai, R. Tang, Y. Wang, Y. Yu, and W. Zhang, “RethinkMCTS: Refining Erroneous Thoughts in Monte Carlo Tree Search for Code Generation,” CoRR , vol. abs/2409.09584, 2024

Original

2024

Earlier work this paper cites.

D. Zhang, X. Huang, D. Zhou, Y. Li, and W. Ouyang, “Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8B,” CoRR , vol. abs/2406.07394, 2024

Original

2024

Earlier work this paper cites.

G. Rabby, F. Keya, P. Zamil, and S. Auer, “MC-NEST - Enhancing Mathematical Reasoning in Large Language Models with a Monte Carlo Nash Equilibrium Self-Refine Tree,” CoRR , vol. abs/2411.15645, 2024

Original

2024

Earlier work this paper cites.

B. Xu, Y. Lin, Y. Li, and Y. Gao, “SRA-MCTS: Self-driven Reasoning Augmentation with Monte Carlo Tree Search for Code Generation,” CoRR , vol. abs/2411.11053, 2024

Original

2024

Earlier work this paper cites.

J. Cheng, X. Liu, C. Wang, X. Gu, Y. Lu, D. Zhang, Y. Dong, J. Tang, H. Wang, and M. Huang, “SPaR: Self-Play with Tree-Search Refinement to Improve Instruction-Following in Large Language Models,” CoRR , vol. abs/2412.11605, 2024

Original

2024

Earlier work this paper cites.

J. Kang, X. Z. Li, X. Chen, A. Kazemi, and B. Chen, “MindStar: Enhancing Math Reasoning in Pre-trained LLMs at Inference Time,” CoRR , vol. abs/2405.16265, 2024

Original

2024

Earlier work this paper cites.

P. Kadam, “GPT-Guided Monte Carlo Tree Search for Symbolic Regression in Financial Fraud Detection,” CoRR , vol. abs/2411.04459, 2024

Original

2024

Earlier work this paper cites.

D. Zhang, J. Wu, J. Lei, T. Che, J. Li, T. Xie, X. Huang, S. Zhang, M. Pavone, Y. Li, W. Ouyang, and D. Zhou, “LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoning,” CoRR , vol. abs/2410.02884, 2024

Original

2024

Earlier work this paper cites.

D. Zhang, S. Zhoubian, Y. Yue, Y. Dong, and J. Tang, “ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search,” CoRR , vol. abs/2406.03816, 2024

Original

2024

Earlier work this paper cites.

H. Yao, J. Huang, W. Wu, J. Zhang, Y. Wang, S. Liu, Y. Wang, Y. Song, H. Feng, L. Shen, and D. Tao, “Mulberry: Empowering mllm with o1-like reasoning and reflection via collective monte carlo tree search,” 2024. [Online]. Available: https://arxiv.org/abs/2412.18319

Original

2024

Earlier work this paper cites.

C. Zhang, J. Song, S. Li, Y. Liang, Y. Ma, W. Wang, Y. Zhu, and S.-C. Zhu, “Proposing and solving olympiad geometry with guided tree search,” 2024. [Online]. Available: https://arxiv.org/abs/2412.10673

Original

2024

Cited alongside, same era.

J. Liu, A. Cohen, R. Pasunuru, Y. Choi, H. Hajishirzi, and A. Celikyilmaz, “Don’t throw away your value model! generating more preferable text with value-guided monte-carlo tree search decoding,” in First Conference on Language Modeling , 2024. [Online]. Available: https://openreview.net/forum?id=kh9Zt2Ldmn

2024

Cited alongside, same era.

Y. Xie, A. Goyal, W. Zheng, M. Kan, T. P. Lillicrap, K. Kawaguchi, and M. Shieh, “Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learning,” CoRR , vol. abs/2405.00451, 2024

Original

2024

Cited alongside, same era.

J. Y. Koh, S. McAleer, D. Fried, and R. Salakhutdinov, “Tree Search for Language Model Agents,” CoRR , vol. abs/2407.01476, 2024

Original

2024

Z. Zhang, C. Zheng, Y. Wu, B. Zhang, R. Lin, B. Yu, D. Liu, J. Zhou, and J. Lin, “The lessons of developing process reward models in mathematical reasoning,” arXiv preprint arXiv:2501.07301 , 2025

Original

2025

Closest in time.

Z. Sun, Q. Wang, W. Yu, X. Zang, K. Zheng, J. Xu, X. Zhang, S. Yang, and H. Li, “ReARTeR: Retrieval-Augmented Reasoning with Trustworthy Process Rewarding,” arXiv preprint arXiv:2501.07861 , 2025

Original

2025

Closest in time.

W. Wang, Z. Gao, L. Chen, Z. Chen, J. Zhu, X. Zhao, Y. Liu, Y. Cao, S. Ye, X. Zhu et al. , “Visualprm: An effective process reward model for multimodal reasoning,” arXiv preprint arXiv:2503.10291 , 2025

Original

2025

Closest in time.

Y. Wang, Y. Zang, H. Li, C. Jin, and J. Wang, “Unified reward model for multimodal understanding and generation,” 2025. [Online]. Available: https://arxiv.org/abs/2503.05236

Original

2025

Closest in time.