Fetching the paper…
Reading the bibliography…
This paper proposes the use of "multicalibration" to yield interpretable and reliable confidence scores for outputs generated by large language models (LLMs).
Verification of forecasts expressed in terms of probability
Brier, G. W · 1950
Earlier work this paper cites.
Perplexity—a measure of the difficulty of speech recognition tasks
Jelinek, F., Mercer, R. L., Bahl, L. R., and Baker, J. K · 1977
Earlier work this paper cites.
Bias plus variance decomposition for zero-one loss functions
Kohavi, R., Wolpert, D. H., et al · 1996
Earlier work this paper cites.
Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods
Platt, J. et al · 1999
Earlier work this paper cites.
Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers
Zadrozny, B. and Elkan, C · 2001
Earlier work this paper cites.
Visualizing data using t-sne
Van der Maaten, L. and Hinton, G · 2008
Earlier work this paper cites.
Bigbench: Towards an industry standard benchmark for big data analytics
Ghazal, A., Rabl, T., Hu, M., Raab, F., Poess, M., Crolotte, A., and Jacobsen, H.-A · 2013
Earlier work this paper cites.
Distribution-free prediction bands for non-parametric regression
Lei, J. and Wasserman, L · 2014
Earlier work this paper cites.
Obtaining well calibrated probabilities using bayesian binning
Naeini, M. P., Cooper, G., and Hauskrecht, M · 2015
Earlier work this paper cites.
On calibration of modern neural networks
Guo, C., Pleiss, G., Sun, Y., and Weinberger, K. Q · 2017
Earlier work this paper cites.
Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension
Joshi, M., Choi, E., Weld, D. S., and Zettlemoyer, L · 2017
Earlier work this paper cites.
Multicalibration: Calibration for the (computationally-identifiable) masses
Hébert-Johnson, U., Kim, M., Reingold, O., and Rothblum, G · 2018
Earlier work this paper cites.
Umap: Uniform manifold approximation and projection for dimension reduction
McInnes, L., Healy, J., and Melville, J · 2018
Earlier work this paper cites.
Can a suit of armor conduct electricity? a new dataset for open book question answering
Mihaylov, T., Clark, P., Khot, T., and Sabharwal, A · 2018
Earlier work this paper cites.
Mathqa: Towards interpretable math word problem solving with operation-based formalisms
Amini, A., Gabriel, S., Lin, P., Koncel-Kedziorski, R., Choi, Y., and Hajishirzi, H · 2019
Earlier work this paper cites.
Multiaccuracy: Black-box post-processing for fairness in classification
Kim, M. P., Ghorbani, A., and Zou, J · 2019
Earlier work this paper cites.
Measuring massive multitask language understanding
Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., and Steinhardt, J · 2020
Earlier work this paper cites.
Evaluating large language models trained on code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. d. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al · 2021
Earlier work this paper cites.
Truthfulqa: Measuring how models mimic human falsehoods
Lin, S., Hilton, J., and Evans, O · 2021
Earlier work this paper cites.
A token-level reference-free hallucination detection benchmark for free-form text generation
Liu, T., Zhang, Y., Brockett, C., Mao, Y., Sui, Z., Chen, W., and Dolan, B · 2021
Earlier work this paper cites.
On hallucination and predictive uncertainty in conditional language generation
Xiao, Y. and Wang, W. Y · 2021
Earlier work this paper cites.
Practical adversarial multivalid conformal prediction
Bastani, O., Gupta, V., Jung, C., Noarov, G., Ramalingam, R., and Roth, A · 2022
Earlier work this paper cites.
Scaling instruction-finetuned language models, 2022
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, E., Wang, X., Dehghani, M., Brahma, S., Webson, A., Gu, S. S., Dai, Z., Suzgun, M., Chen, X., Chowdhery, A., Narang, S., Mishra, G., Yu, A., Zhao, V., Huang, Y., Dai, A., Yu, H., Petrov, S., Chi, E. H., Dean, J., Devlin, J., Roberts, A., Zhou, D., Le, Q. V., and Wei, J · 2022
Earlier work this paper cites.
Domino: Discovering systematic errors with cross-modal embeddings
Eyuboglu, S., Varma, M., Saab, K., Delbrouck, J.-B., Lee-Messer, C., Dunnmon, J., Zou, J., and Ré, C · 2022
Cited alongside, same era.
Batch multivalid conformal prediction
Jung, C., Noarov, G., Ramalingam, R., and Roth, A · 2022
Cited alongside, same era.
Language models (mostly) know what they know
Kadavath, S., Conerly, T., Askell, A., Henighan, T., Drain, D., Perez, E., Schiefer, N., Hatfield-Dodds, Z., DasSarma, N., Tran-Johnson, E., et al · 2022
Cited alongside, same era.
Crosslingual generalization through multitask finetuning
Muennighoff, N., Wang, T., Sutawika, L., Roberts, A., Biderman, S., Scao, T. L., Bari, M. S., Shen, S., Yong, Z.-X., Schoelkopf, H., et al · 2022
Cited alongside, same era.
Angle-optimized text embeddings
Li, X. and Li, J · 2023
Later among the works it cites.
Generating with confidence: Uncertainty quantification for black-box large language models
Lin, Z., Trivedi, S., and Sun, J · 2023
Later among the works it cites.
Calibrating llm-based evaluator
Liu, Y., Yang, T., Huang, S., Zhang, Z., Huang, H., Wei, F., Deng, W., Sun, F., and Zhang, Q · 2023
Later among the works it cites.
Zero-resource hallucination prevention for large language models
Luo, J., Xiao, C., and Ma, F · 2023
Later among the works it cites.
Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Roth, A · 2022
Cited alongside, same era.
Uncertainty in natural language generation: From theory to applications
Baan, J., Daheim, N., Ilia, E., Ulmer, D., Li, H.-S., Fernández, R., Plank, B., Sennrich, R., Zerva, C., and Aziz, W · 2023
Cited alongside, same era.
A survey on evaluation of large language models
Chang, Y., Wang, X., Wang, J., Wu, Y., Zhu, K., Chen, H., Yang, L., Yi, X., Wang, C., Wang, Y., et al · 2023
Cited alongside, same era.
Chen, J. and Mueller, J · 2023
Cited alongside, same era.
Hallucination detection: Robustly discerning reliable answers in large language models
Chen, Y., Fu, Q., Yuan, Y., Wen, Z., Fan, G., Liu, D., Zhang, D., Li, Z., and Xiao, Y · 2023
Cited alongside, same era.
Conformal autoregressive generation: Beam search with coverage guarantees
Deutschmann, N., Alberts, M., and Martínez, M. R · 2023
Cited alongside, same era.
Shifting attention to relevance: Towards the uncertainty estimation of large language models
Duan, J., Cheng, H., Wang, S., Wang, C., Zavalny, A., Xu, R., Kailkhura, B., and Xu, K · 2023
Cited alongside, same era.
Halo: Estimation and reduction of hallucinations in open-source weak large language models
Elaraby, M., Lu, M., Dunn, J., Zhang, X., Wang, Y., and Liu, S · 2023
Cited alongside, same era.
Manakul, P., Liusie, A., and Gales, M. J · 2023
Later among the works it cites.
Orca: Progressive learning from complex explanation traces of gpt-4, 2023
Mukherjee, S., Mitra, A., Jawahar, G., Agarwal, S., Palangi, H., and Awadallah, A · 2023
Later among the works it cites.
High-dimensional prediction for sequential decision making
Noarov, G., Ramalingam, R., Roth, A., and Xie, S · 2023
Later among the works it cites.
Quach, V., Fisch, A., Schuster, T., Yala, A., Sohn, J. H., Jaakkola, T. S., and Barzilay, R · 2023
Later among the works it cites.
A survey of hallucination in large foundation models
Rawte, V., Sheth, A., and Das, A · 2023
Later among the works it cites.
Nemo guardrails: A toolkit for controllable and safe llm applications with programmable rails
Rebedea, T., Dinu, R., Sreedhar, M., Parisien, C., and Cohen, J · 2023
Later among the works it cites.
Robots that ask for help: Uncertainty alignment for large language model planners
Ren, A. Z., Dixit, A., Bodrova, A., Singh, S., Tu, S., Brown, N., Xu, P., Takayama, L., Xia, F., Varley, J., et al · 2023
Later among the works it cites.
Tian, K., Mitchell, E., Zhou, A., Sharma, A., Rafailov, R., Yao, H., Finn, C., and Manning, C. D · 2023
Later among the works it cites.
Llama 2: Open foundation and fine-tuned chat models, 2023
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., Bikel, D., Blecher, L., Ferrer, C. C., Chen, M., Cucurull, G., Esiobu, D., Fernandes, J., Fu, J., Fu, W., Fuller, B., Gao, C., Goswami, V., Goyal, N., Hartshorn, A., Hosseini, S., Hou, R., Inan, H., Kardas, M., Kerkez, V., Khabsa, M., Kloumann, I., Korenev, A., Koura, P. S., Lachaux, M.-A., Lavril, T., Lee, J., Liskovich, D., Lu, Y., Mao, Y., Martinet, X., Mihaylov, T., Mishra, P., Molybog, I., Nie, Y., Poulton, A., Reizenstein, J., Rungta, R., Saladi, K., Schelten, A., Silva, R., Smith, E. M., Subramanian, R., Tan, X. E., Tang, B., Taylor, R., Williams, A., Kuan, J. X., Xu, P., Yan, Z., Zarov, I., Zhang, Y., Fan, A., Kambadur, M., Narang, S., Rodriguez, A., Stojnic, R., Edunov, S., and Scialom, T · 2023
Later among the works it cites.
Varshney, N., Yao, W., Zhang, H., Chen, J., and Yu, D · 2023
Later among the works it cites.
Reducing llm hallucinations using epistemic neural networks
Verma, S., Tran, K., Ali, Y., and Min, G · 2023
Later among the works it cites.
Uncertainty-aware language modeling for selective question answering
Yang, Q., Ravikumar, S., Schmitt-Ulms, F., Lolla, S., Demir, E., Elistratov, I., Lavaee, A., Lolla, S., Ahmadi, E., Rus, D., et al · 2023
Later among the works it cites.
Llm lies: Hallucinations are not bugs, but features as adversarial examples
Yao, J.-Y., Ning, K.-P., Liu, Z.-H., Ning, M.-N., and Yuan, L · 2023
Later among the works it cites.
Cognitive mirage: A review of hallucinations in large language models
Ye, H., Liu, T., Zhang, A., Hua, W., and Jia, W · 2023
Later among the works it cites.
Zecchin, M., Park, S., and Simeone, O · 2023
Later among the works it cites.
Zhao, T., Wei, M., Preston, J. S., and Poon, H · 2023
Later among the works it cites.
Large language models for information retrieval: A survey
Zhu, Y., Yuan, H., Wang, S., Liu, J., Liu, W., Deng, C., Dou, Z., and Wen, J.-R · 2023
Later among the works it cites.
A comprehensive survey of hallucination mitigation techniques in large language models
Tonmoy, S., Zaman, S., Jain, V., Rani, A., Rawte, V., Chadha, A., and Das, A · 2024
Closest in time.