Are Large Language Models More Honest in Their Probabilistic or Verbalized Confidence?

Abdi, H.: The kendall rank correlation coefficient. Encyclopedia of measurement and statistics 2

2007

Earlier work this paper cites.

Hauke, J., Kossowski, T.: Comparison of values of pearson’s and spearman’s correlation coefficients on the same sets of data. Quaestiones geographicae 30

2011

Earlier work this paper cites.

Guo, C., Pleiss, G., Sun, Y., Weinberger, K.Q.: On calibration of modern neural networks. In: International conference on machine learning. pp. 1321–1330. PMLR (2017)

2017

Earlier work this paper cites.

Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

Original

2018

Earlier work this paper cites.

Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al.: Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics 7

2019

Earlier work this paper cites.

Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al.: Language models are few-shot learners. Advances in neural information processing systems 33

2020

Earlier work this paper cites.

Desai, S., Durrett, G.: Calibration of pre-trained transformers. arXiv preprint arXiv:2003.07892 (2020)

Original

2020

Earlier work this paper cites.

Karpukhin, V., Oğuz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., Yih, W.t.: Dense passage retrieval for open-domain question answering. arXiv preprint arXiv:2004.04906 (2020)

Original

2020

Earlier work this paper cites.

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.t., Rocktäschel, T., et al.: Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems 33

2020

Cited alongside, same era.

Jiang, Z., Araki, J., Ding, H., Neubig, G.: How can we know when language models know? on the calibration of language models for question answering. Transactions of the Association for Computational Linguistics 9

2021

Cited alongside, same era.

Kadavath, S., Conerly, T., Askell, A., Henighan, T., Drain, D., Perez, E., Schiefer, N., Hatfield-Dodds, Z., DasSarma, N., Tran-Johnson, E., et al.: Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221 (2022)

Original

2022

Cited alongside, same era.

Lin, S., Hilton, J., Evans, O.: Teaching models to express their uncertainty in words. arXiv preprint arXiv:2205.14334 (2022)

Original

2022

Cited alongside, same era.

Ni, S., Bi, K., Guo, J., Cheng, X.: A comparative study of training objectives for clarification facet generation. In: Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region. pp. 1–10 (2023)

2023

Later among the works it cites.

Tian, K., Mitchell, E., Zhou, A., Sharma, A., Rafailov, R., Yao, H., Finn, C., Manning, C.D.: Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback. arXiv preprint arXiv:2305.14975 (2023)

Original

2023

Later among the works it cites.

Xiong, M., Hu, Z., Lu, X., Li, Y., Fu, J., He, J., Hooi, B.: Can llms express their uncertainty? an empirical evaluation of confidence elicitation in llms. arXiv preprint arXiv:2306.13063 (2023)

Original

2023

Later among the works it cites.

Yang, Y., Chern, E., Qiu, X., Neubig, G., Liu, P.: Alignment for honesty. arXiv preprint arXiv:2312.07000 (2023)

Original

2023