Internal Consistency and Self-Feedback in Large Language Models: A Survey

A. Tarski, Introduction to logic: And to the methodology of deductive sciences . Oxford University Press, 1941

1941

Earlier work this paper cites.

M. Jin, Q. Yu et al. , “The impact of reasoning step length on large language models,” in Proc. of ACL Findings , 2024, pp. 1830–1842

1999

Earlier work this paper cites.

I. Harvey, “The microbial genetic algorithm,” in Advances in Artificial Life. Darwin Meets von Neumann , 2011, pp. 126–133

2011

Earlier work this paper cites.

Y. Gal and Z. Ghahramani, “Dropout as a bayesian approximation: Representing model uncertainty in deep learning,” in Proc. of ICML , 2016, pp. 1050–1059

2016

Earlier work this paper cites.

J. Schulman, F. Wolski et al. , “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347 , 2017

Original

2017

Earlier work this paper cites.

P. Clark, I. Cowhey et al. , “Think you have solved question answering? try arc, the ai2 reasoning challenge,” arXiv preprint arXiv:1803.05457 , 2018

Original

2018

Earlier work this paper cites.

A. Ignatiev, A. Morgado, and J. Marques-Silva, “RC2: An Efficient MaxSAT Solver,” Journal on Satisfiability, Boolean Modeling and Computation , pp. 53–64, 2019

2019

Earlier work this paper cites.

M. T. Pilehvar and J. Camacho-Collados, “Wic: the word-in-context dataset for evaluating context-sensitive meaning representations,” Proceedings of NAACL 2019 (short) , 2019

2019

Earlier work this paper cites.

J. Kaplan, S. McCandlish et al. , “Scaling laws for neural language models,” arXiv preprint arXiv:2001.08361 , 2020

Original

2020

Earlier work this paper cites.

D. Hendrycks, C. Burns et al. , “Measuring massive multitask language understanding,” in Proc. of ICLR , 2021

2021

Earlier work this paper cites.

K. Cobbe, V. Kosaraju et al. , “Training verifiers to solve math word problems,” arXiv preprint arXiv:2110.14168 , 2021

Original

2021

Earlier work this paper cites.

E. M. Bender, T. Gebru et al. , “On the dangers of stochastic parrots: Can language models be too big?” in Proceedings of the 2021 ACM conference on fairness, accountability, and transparency , 2021, pp. 610–623

2021

Earlier work this paper cites.

Y. Xiao and W. Y. Wang, “On hallucination and predictive uncertainty in conditional language generation,” in Proc. of EACL , 2021, pp. 2734–2744

2021

Earlier work this paper cites.

W. Yuan, G. Neubig, and P. Liu, “BARTScore: Evaluating generated text as text generation,” in Proc. of NeurIPS , 2021

2021

Earlier work this paper cites.

Y. Elazar, N. Kassner et al. , “Measuring and improving consistency in pretrained language models,” TACL , pp. 1012–1031, 2021

2021

Earlier work this paper cites.

M. Chen, J. Tworek et al. , “Evaluating large language models trained on code,” arXiv preprint arXiv:2107.03374 , 2021

Original

2021

Earlier work this paper cites.

D. Hendrycks, C. Burns et al. , “Measuring mathematical problem solving with the math dataset,” NeurIPS , 2021

2021

Earlier work this paper cites.

J. Wei, X. Wang et al. , “Chain of thought prompting elicits reasoning in large language models,” in Proc. of NeurIPS , 2022

2022

Earlier work this paper cites.

S. Lin, J. Hilton, and O. Evans, “TruthfulQA: Measuring how models mimic human falsehoods,” in Proc. of ACL , 2022, pp. 3214–3252

2022

Earlier work this paper cites.

Y. Ma, D. Tsao, and H.-Y. Shum, “On the principles of parsimony and self-consistency for the emergence of intelligence,” Frontiers of Information Technology & Electronic Engineering , pp. 1298–1323, 2022

2022

Earlier work this paper cites.

S. Kadavath, T. Conerly et al. , “Language models (mostly) know what they know,” arXiv preprint arXiv:2207.05221 , 2022

Original

2022

Earlier work this paper cites.

E. Mitchell, J. Noh et al. , “Enhancing self-consistency and performance of pre-trained language models through natural language inference,” in Proc. of EMNLP , 2022, pp. 1754–1768

2022

Earlier work this paper cites.

W. Saunders, C. Yeh et al. , “Self-critiquing models for assisting human evaluators,” arXiv preprint arXiv:2206.05802 , 2022

Original

2022

Earlier work this paper cites.

J. Jung, L. Qin et al. , “Maieutic prompting: Logically consistent reasoning with recursive explanations,” in Proc. of EMNLP , 2022, pp. 1266–1279

2022

Earlier work this paper cites.

A. Agarwal, A. Tzen, and C. Tew, “Improving logical consistency in pre-trained language models using natural language inference,” 2022

2022

Earlier work this paper cites.

K. Yang, Y. Tian et al. , “Re3: Generating longer stories with recursive reprompting and revision,” in Proc. of EMNLP , 2022, pp. 4393–4479

2022

Earlier work this paper cites.

X. L. Li, A. Holtzman et al. , “Contrastive decoding: Open-ended text generation as optimization,” arXiv preprint arXiv:2210.15097 , 2022

Original

2022

Earlier work this paper cites.

Y. Bai, S. Kadavath et al. , “Constitutional ai: Harmlessness from ai feedback,” arXiv preprint arXiv:2212.08073 , 2022

Original

2022

Earlier work this paper cites.

K. Ethayarajh, Y. Choi, and S. Swayamdipta, “Understanding dataset difficulty with 𝒱 \mathcal{V} -usable information,” in Proc. of ICML , 2022, pp. 5988–6008

2022

Earlier work this paper cites.

L. Ouyang, J. Wu et al. , “Training language models to follow instructions with human feedback,” Proc. of NeurIPS , pp. 27 730–27 744, 2022

2022

Earlier work this paper cites.

M. Jang, D. S. Kwon, and T. Lukasiewicz, “BECEL: Benchmark for consistency evaluation of language models,” in Proc. of COLING , 2022, pp. 3680–3696

2022

Earlier work this paper cites.

M. Suzgun, N. Scales et al. , “Challenging big-bench tasks and whether chain-of-thought can solve them,” arXiv preprint arXiv:2210.09261 , 2022

Original

2022

Earlier work this paper cites.

W. X. Zhao, K. Zhou et al. , “A survey of large language models,” arXiv preprint arXiv:2303.18223 , 2023

Original

2023

Earlier work this paper cites.

X. Wang, J. Wei et al. , “Self-consistency improves chain of thought reasoning in language models,” in Proc. of ICLR , 2023

2023

Earlier work this paper cites.

Z. Yin, Q. Sun et al. , “Do large language models know what they don’t know?” in Proc. of ACL Findings , 2023, pp. 8653–8665

2023

Earlier work this paper cites.

K. Li, O. Patel et al. , “Inference-time intervention: Eliciting truthful answers from a language model,” in Proc. of NeurIPS , 2023

2023

Earlier work this paper cites.

A. Madaan, N. Tandon et al. , “Self-refine: Iterative refinement with self-feedback,” in Proc. of NeurIPS , 2023

2023

Earlier work this paper cites.

S. Welleck, X. Lu et al. , “Generating sequences by learning to self-correct,” in Proc. of ICLR , 2023

2023

Earlier work this paper cites.

Y. Zhang, Y. Li et al. , “Siren’s song in the ai ocean: a survey on hallucination in large language models,” arXiv preprint arXiv:2309.01219 , 2023

Original

2023

Earlier work this paper cites.

Y. Gao, Y. Xiong et al. , “Retrieval-augmented generation for large language models: A survey,” arXiv preprint arXiv:2312.10997 , 2023

Original

2023

Earlier work this paper cites.

M. Hu, Z. Zhang et al. , “Uncertainty in natural language processing: Sources, quantification, and applications,” arXiv preprint arXiv:2306.04459 , 2023

Original

2023

Earlier work this paper cites.

B. Liu, J. T. Ash et al. , “Exposing attention glitches with flip-flop language modeling,” in Proc. of NeurIPS , 2023

2023

Earlier work this paper cites.

M. Zhang, O. Press et al. , “How language model hallucinations can snowball,” arXiv preprint arXiv:2305.13534 , 2023

Original

2023

Earlier work this paper cites.

S. Yao, D. Yu et al. , “Tree of thoughts: Deliberate problem solving with large language models,” in Proc. of NeurIPS , 2023

2023

Earlier work this paper cites.

J. Huang, S. Gu et al. , “Large language models can self-improve,” in Proc. of EMNLP , 2023, pp. 1051–1068

2023

Earlier work this paper cites.

S. An, Z. Ma et al. , “Learning from mistakes makes llm better reasoner,” arXiv preprint arXiv:2310.20689 , 2023

Original

2023

Earlier work this paper cites.

K. Xiong, X. Ding et al. , “Examining inter-consistency of large language models collaboration: An in-depth analysis via debate,” in Proc. of EMNLP Findings , 2023, pp. 7572–7590

2023

Earlier work this paper cites.

Y. Du, S. Li et al. , “Improving factuality and reasoning in language models through multiagent debate,” arXiv preprint arXiv:2305.14325 , 2023

Original

2023

Earlier work this paper cites.

N. Shinn, F. Cassano et al. , “Reflexion: language agents with verbal reinforcement learning,” in Proc. of NeurIPS , 2023, pp. 8634–8652

2023

Cited alongside, same era.

H. Kang, J. Ni, and H. Yao, “Ever: Mitigating hallucination in large language models through real-time verification and rectification,” arXiv preprint arXiv:2311.09114 , 2023

Original

2023

Cited alongside, same era.

W. Shi, X. Han et al. , “Trusting your evidence: Hallucinate less with context-aware decoding,” arXiv preprint arXiv:2305.14739 , 2023

Original

2023

Cited alongside, same era.

D. Deng, G. Chen et al. , “Uncertainty estimation by fisher information-based evidential deep learning,” in Proc. of ICML , 2023, pp. 7596–7616

2023

Cited alongside, same era.

S. Diao, P. Wang et al. , “Active prompting with chain-of-thought for large language models,” arXiv preprint arXiv:2302.12246 , 2023

M. Besta, N. Blach et al. , “Graph of thoughts: Solving elaborate problems with large language models,” Proc. of AAAI , pp. 17 682–17 690, 2024

2024

Closest in time.

E. Zelikman, G. Harik et al. , “Quiet-star: Language models can teach themselves to think before speaking,” arXiv preprint arXiv:2403.09629 , 2024

Original

2024

Closest in time.

Y. Tong, D. Li et al. , “Can llms learn from previous mistakes? investigating llms’ errors to boost for reasoning,” arXiv preprint arXiv:2403.20046 , 2024

Original

2024

Closest in time.

C. Qian, Z. Xie et al. , “Scaling large-language-model-based multi-agent collaboration,” arXiv preprint arXiv:2406.07155 , 2024

Original

2024

Closest in time.

D. Paul, M. Ismayilzada et al. , “REFINER: Reasoning feedback on intermediate representations,” in Proc. of EACL , 2024, pp. 1100–1126