Advancing Reasoning in Large Language Models: Promising Methods and Approaches

R. R. Yager, “Approximate reasoning as a basis for rule-based expert systems,” IEEE Transactions on Systems, Man, and Cybernetics , no. 4, pp. 636–643, 1984

1984

Earlier work this paper cites.

R. Sun, “Robust reasoning: integrating rule-based and similarity-based reasoning,” Artificial Intelligence , vol. 75, no. 2, pp. 241–295, 1995

1995

Earlier work this paper cites.

Q. Liu, H. Jiang, A. Evdokimov, Z.-H. Ling, X. Zhu, S. Wei, and Y. Hu, “Probabilistic reasoning via deep learning: Neural association models,” arXiv preprint arXiv:1603.07704 , 2016

Original

2016

Earlier work this paper cites.

A. Santoro, D. Raposo, D. G. Barrett, M. Malinowski, R. Pascanu, P. Battaglia, and T. Lillicrap, “A simple neural network module for relational reasoning,” in Advances in Neural Information Processing Systems , I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds., vol. 30. Curran Associates, Inc., 2017

2017

Earlier work this paper cites.

W. L. Hamilton et al. , “Inductive representation learning on large graphs,” Advances in Neural Information Processing Systems , 2017

2017

Earlier work this paper cites.

C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger, “On calibration of modern neural networks,” in International Conference on Machine Learning (ICML) , 2017, pp. 1321–1330

2017

Earlier work this paper cites.

Z. C. Lipton, “The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery.” Queue , vol. 16, no. 3, pp. 31–57, 2018

2018

Earlier work this paper cites.

R. Zellers, Y. Bisk, R. Schwartz, and Y. Choi, “Swag: A large-scale adversarial dataset for grounded commonsense inference,” arXiv preprint arXiv:1808.05326 , 2018

Original

2018

Earlier work this paper cites.

P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord, “Think you have solved question answering? try arc, the ai2 reasoning challenge,” arXiv preprint arXiv:1803.05457 , 2018

Original

2018

Earlier work this paper cites.

Z. Yang, P. Qi, S. Zhang, Y. Bengio, W. Cohen, R. Salakhutdinov, and C. D. Manning, “Hotpotqa: A dataset for diverse, explainable multi-hop question answering,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , 2018, pp. 2369–2380

2018

Earlier work this paper cites.

B. Lake and M. Baroni, “Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks,” in International conference on machine learning . PMLR, 2018, pp. 2873–2882

2018

Earlier work this paper cites.

C. Bhagavatula, R. L. Bras, C. Malaviya, K. Sakaguchi, A. Holtzman, H. Rashkin, D. Downey, S. W.-t. Yih, and Y. Choi, “Abductive commonsense reasoning,” arXiv preprint arXiv:1908.05739 , 2019

Original

2019

Earlier work this paper cites.

T. Brown et al. , “Language models are few-shot learners,” Advances in Neural Information Processing Systems , 2020

2020

Earlier work this paper cites.

X. Zhou, Y. Zhang, L. Cui, and D. Huang, “Evaluating commonsense in pre-trained language models,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 34, no. 05, 2020, pp. 9733–9740

2020

Earlier work this paper cites.

P. Lewis et al. , “Retrieval-augmented generation for knowledge-intensive nlp tasks,” Advances in Neural Information Processing Systems , 2020

2020

Earlier work this paper cites.

A. Talmor, O. Tafjord, P. Clark, Y. Goldberg, and J. Berant, “Leap-of-thought: Teaching pre-trained models to systematically reason over implicit knowledge,” Advances in Neural Information Processing Systems , vol. 33, pp. 20 227–20 237, 2020

2020

Earlier work this paper cites.

C. Durkan, I. Murray, and G. Papamakarios, “On contrastive learning for likelihood-free inference,” in International conference on machine learning . PMLR, 2020, pp. 2771–2781

2020

Earlier work this paper cites.

Y. Nie, A. Williams, E. Dinan, M. Bansal, J. Weston, and D. Kiela, “Adversarial nli: A new benchmark for natural language understanding,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics . Association for Computational Linguistics, 2020

2020

Cited alongside, same era.

D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt, “Measuring massive multitask language understanding,” arXiv preprint arXiv:2009.03300 , 2020

Original

2020

Cited alongside, same era.

P. Clark, O. Tafjord, and K. Richardson, “Transformers as soft reasoners over language,” in Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence , 2021, pp. 3882–3890

2021

Cited alongside, same era.

K. Shuster, S. Poff, M. Chen, D. Kiela, and J. Weston, “Retrieval augmentation reduces hallucination in conversation,” in Findings of the Association for Computational Linguistics: EMNLP 2021 , 2021, pp. 3784–3803

L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al. , “Training language models to follow instructions with human feedback,” Advances in neural information processing systems , vol. 35, pp. 27 730–27 744, 2022

2022

Later among the works it cites.

A. Srivastava et al. , “Beyond the imitation game: Quantifying and extrapolating the capabilities of language models,” arXiv preprint arXiv:2206.04615 , 2022

Original

2022

Later among the works it cites.

A. d. Garcez and L. C. Lamb, “Neurosymbolic ai: The 3 rd wave,” Artificial Intelligence Review , vol. 56, no. 11, pp. 12 387–12 406, 2023

2023

Later among the works it cites.

L. Gao, A. Madaan, S. Zhou, U. Alon, P. Liu, Y. Yang, J. Callan, and G. Neubig, “Pal: Program-aided language models,” in International Conference on Machine Learning . PMLR, 2023, pp. 10 764–10 799

2023

Later among the works it cites.