UTMath: Math Evaluation with Unit Test via Reasoning-to-Coding Thoughts

Cognitive tutor: Applied research in mathematics education

Steven Ritter, John R Anderson, Kenneth R Koedinger, and Albert Corbett. 2007 · 2007

Earlier work this paper cites.

Evaluating large language models trained on code

Original

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021 · 2021

Earlier work this paper cites.

Training verifiers to solve math word problems

Original

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021 · 2021

Earlier work this paper cites.

Measuring mathematical problem solving with the math dataset

Original

Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021 · 2021

Earlier work this paper cites.

Are nlp models really able to solve simple math word problems?

Original

Arkil Patel, Satwik Bhattamishra, and Navin Goyal. 2021 · 2021

Earlier work this paper cites.

Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks

Original

Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W Cohen. 2022 · 2022

Earlier work this paper cites.

" john is 50 years old, can his son be 65?" evaluating nlp models’ understanding of feasibility

Original

Himanshu Gupta, Neeraj Varshney, Swaroop Mishra, Kuntal Kumar Pal, Saurabh Arjun Sawant, Kevin Scaria, Siddharth Goyal, and Chitta Baral. 2022 · 2022

Earlier work this paper cites.

Large language models are zero-shot reasoners

Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022 · 2022

Earlier work this paper cites.

Discovering language model behaviors with model-written evaluations

Original

Ethan Perez, Sam Ringer, Kamilė Lukošiūtė, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, et al. 2022 · 2022

Earlier work this paper cites.

Large language models still can’t plan (a benchmark for llms on planning and reasoning about change)

Karthik Valmeekam, Alberto Olmo, Sarath Sreedharan, and Subbarao Kambhampati. 2022 · 2022

Earlier work this paper cites.

Chain-of-thought prompting elicits reasoning in large language models

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022 · 2022

Earlier work this paper cites.

Rewriting math word problems to improve learning outcomes for emerging readers: a randomized field trial in carnegie learning’s mathia

Husni Almoubayyed, Rae Bastoni, Susan R Berman, Sarah Galasso, Megan Jensen, Leila Lester, April Murphy, Mark Swartz, Kyle Weldon, Stephen E Fancsali, et al. 2023 · 2023

Earlier work this paper cites.

Theoremqa: A theorem-driven question answering dataset

Wenhu Chen, Ming Yin, Max Ku, Pan Lu, Yixin Wan, Xueguang Ma, Jianyu Xu, Xinyi Wang, and Tony Xia. 2023 · 2023

Earlier work this paper cites.

Can large language models be an alternative to human evaluations?

Original

Cheng-Han Chiang and Hung-yi Lee. 2023 · 2023

Earlier work this paper cites.

From human days to machine seconds: Automatically answering and generating machine learning final exams

Iddo Drori, Sarah J Zhang, Reece Shuttleworth, Sarah Zhang, Keith Tyser, Zad Chin, Pedro Lantigua, Saisamrit Surbehera, Gregory Hunter, Derek Austin, et al. 2023 · 2023

Cited alongside, same era.

Chain-of-thought hub: A continuous effort to measure large language models’ reasoning performance

Original

Yao Fu, Litu Ou, Mingyu Chen, Yuhao Wan, Hao Peng, and Tushar Khot. 2023 · 2023

Cited alongside, same era.

Pal: Program-aided language models

Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023 · 2023

Cited alongside, same era.

Large language models are state-of-the-art evaluators of translation quality

Original

Tom Kocmi and Christian Federmann. 2023 · 2023

Cited alongside, same era.

Do you really follow me? adversarial instructions for evaluating the robustness of large language models

Large language models for mathematical reasoning: Progresses and challenges

Original

Janice Ahn, Rishu Verma, Renze Lou, Di Liu, Rui Zhang, and Wenpeng Yin. 2024 · 2024

Closest in time.

Investigating the robustness of llms on math word problems

Original

Ujjwala Anantheswaran, Himanshu Gupta, Kevin Scaria, Shreyas Verma, Chitta Baral, and Swaroop Mishra. 2024 · 2024

Closest in time.

Graph of thoughts: Solving elaborate problems with large language models

Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, et al. 2024 · 2024

Closest in time.

Deepseek llm: Scaling open-source language models with longtermism

Original

Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, et al. 2024 · 2024

Closest in time.

A survey on evaluation of large language models