The Benefits of a Concise Chain of Thought on Problem-Solving in Large Language Models

R. Zellers, A. Holtzman, Y. Bisk, A. Farhadi, and Y. Choi, “Hellaswag: Can a machine really finish your sentence?” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , 2019. [Online]. Available: https://arxiv.org/abs/1905.07830

Original

1905

Earlier work this paper cites.

H. B. Mann and D. R. Whitney, “On a test of whether one of two random variables is stochastically larger than the other,” The Annals of Mathematical Statistics , vol. 18, pp. 50–60, 3 1947. [Online]. Available: https://www.jstor.org/stable/2236101

1947

Earlier work this paper cites.

J. Liu, L. Cui, H. Liu, D. Huang, Y. Wang, and Y. Zhang, “Logiqa: A challenge dataset for machine reading comprehension with logical reasoning,” in International Joint Conference on Artificial Intelligence , 2020. [Online]. Available: https://arxiv.org/abs/2007.08124

Original

2007

Earlier work this paper cites.

P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord, “Think you have solved question answering? try arc, the ai2 reasoning challenge,” ArXiv , 3 2018. [Online]. Available: https://arxiv.org/abs/1803.05457

Original

2018

Earlier work this paper cites.

T. Brown et al. , “Language models are few-shot learners,” in Advances in Neural Information Processing Systems , vol. 33. Curran Associates, Inc., 2020, pp. 1877–1901. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf

2020

Earlier work this paper cites.

T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, “Large language models are zero-shot reasoners,” in Advances in Neural Information Processing Systems , vol. 35, 5 2022, pp. 22 199–22 213. [Online]. Available: https://arxiv.org/abs/2205.11916

Original

2022

Earlier work this paper cites.

J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. Le, and D. Zhou, “Chain-of-thought prompting elicits reasoning in large language models,” arXiv , 1 2022. [Online]. Available: https://arxiv.org/abs/2201.11903

Original

2022

Earlier work this paper cites.

A. Madaan and A. Yazdanbakhsh, “Text and patterns: For effective chain of thought, it takes two to tango,” arXiv , 9 2022. [Online]. Available: https://arxiv.org/abs/2209.07686

Original

2022

Earlier work this paper cites.

A. Pal, L. K. Umapathi, and M. Sankarasubbu, “Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering,” in Proceedings of the Conference on Health, Inference, and Learning . PMLR, 2022, pp. 248–260. [Online]. Available: https://proceedings.mlr.press/v174/pal22a.html

2022

Earlier work this paper cites.

S. Wang, Z. Liu, W. Zhong, M. Zhou, Z. Wei, Z. Chen, and N. Duan, “From lsat: The progress and challenges of complex reasoning,” IEEE/ACM Transactions on Audio, Speech and Language Processing , vol. 30, pp. 2201–2216, 8 2021. [Online]. Available: https://doi.org/10.1109/TASLP.2022.3164218

2022

Cited alongside, same era.

G. Mialon, R. Dessì, M. Lomeli, C. Nalmpantis, R. Pasunuru, R. Raileanu, B. Rozière, T. Schick, J. Dwivedi-Yu, A. Celikyilmaz, E. Grave, Y. LeCun, and T. Scialom, “Augmented language models: a survey,” arXiv , 2 2023. [Online]. Available: https://arxiv.org/abs/2302.07842v1

Original

2023

Cited alongside, same era.

J. White, Q. Fu, S. Hays, M. Sandborn, C. Olea, H. Gilbert, A. Elnashar, J. Spencer-Smith, and D. C. Schmidt, “A prompt pattern catalog to enhance prompt engineering with chatgpt,” arXiv , 2 2023. [Online]. Available: https://arxiv.org/abs/2302.11382

Original

2023

Cited alongside, same era.

Y. Zhou, A. I. Muresanu, Z. Han, K. Paster, S. Pitis, H. Chan, and J. Ba, “Large language models are human-level prompt engineers,” The Eleventh International Conference on Learning Representations , 11 2023. [Online]. Available: https://arxiv.org/abs/2211.01910https://openreview.net/forum?id=92gvk82DE-

W. Zhong, R. Cui, Y. Guo, Y. Liang, S. Lu, Y. Wang, A. Saied, W. Chen, and N. Duan, “Agieval: A human-centric benchmark for evaluating foundation models,” ArXiv , 4 2023. [Online]. Available: https://arxiv.org/abs/2304.06364

Original

2023

Later among the works it cites.

OpenAI, “Introducing chatgpt,” 11 2022, accessed: 2023-04-29. [Online]. Available: https://openai.com/blog/chatgpt

2023

Later among the works it cites.

——, “Gpt-4 technical report,” arXiv , 3 2023, accessed: 2023-04-29. [Online]. Available: https://arxiv.org/abs/2303.08774

Original

2023

Later among the works it cites.

——, “Gpt-4,” 3 2023, accessed: 2023-04-29. [Online]. Available: https://openai.com/research/gpt-4

2023

Later among the works it cites.

Microsoft, “Azure openai service,” 2023, accessed: 2023-12-07. [Online]. Available: https://azure.microsoft.com/en-us/products/ai-services/openai-service/