Beyond the Final Layer: Intermediate Representations for Better Multilingual Calibration in Large Language Models

Do we need language-specific fact-checking models? the case of Chinese

Caiqi Zhang, Zhijiang Guo, and Andreas Vlachos. 2024a · 1914

Earlier work this paper cites.

Verification of forecasts expressed in terms of probability

Glenn W Brier. 1950 · 1950

Earlier work this paper cites.

The interpretation of interaction in contingency tables

E. H. Simpson. 1951 · 1951

Earlier work this paper cites.

Transforming classifier scores into accurate multiclass probability estimates

Bianca Zadrozny and Charles Elkan. 2002 · 2002

Earlier work this paper cites.

An introduction to roc analysis

Tom Fawcett. 2006 · 2006

Earlier work this paper cites.

On calibration of modern neural networks

Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017 · 2017

Earlier work this paper cites.

BERT rediscovers the classical NLP pipeline

Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019 · 2019

Earlier work this paper cites.

Depth-adaptive transformer

Maha Elbayad, Jiatao Gu, Edouard Grave, and Michael Auli. 2020 · 2020

Earlier work this paper cites.

Measuring massive multitask language understanding

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021 · 2021

Earlier work this paper cites.

MKQA: A linguistically diverse benchmark for multilingual open domain question answering

Shayne Longpre, Yi Lu, and Joachim Daiber. 2021 · 2021

Earlier work this paper cites.

Revisiting the calibration of modern neural networks

Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucic. 2021 · 2021

Earlier work this paper cites.

Machine translationese: Effects of algorithmic bias on linguistic complexity in machine translation

Eva Vanmassenhove, Dimitar Shterionov, and Matthew Gwilliam. 2021 · 2021

Earlier work this paper cites.

On the calibration of massively multilingual language models

Kabir Ahuja, Sunayana Sitaram, Sandipan Dandapat, and Monojit Choudhury. 2022 · 2022

Earlier work this paper cites.

A comparative study of confidence calibration in deep learning: From computer vision to medical imaging

Original

Riqiang Gao, Thomas Li, Yucheng Tang, Zhoubing Xu, Michael Kammer, Sanja L. Antic, Kim Sandler, Fabien Moldonado, Thomas A. Lasko, and Bennett Landman. 2022 · 2022

Earlier work this paper cites.

Language models (mostly) know what they know

Original

Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer El-Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav Fort, and 17 others. 2022 · 2022

Earlier work this paper cites.

Teaching models to express their uncertainty in words

Original

Stephanie Lin, Jacob Hilton, and Owain Evans. 2022 · 2022

Earlier work this paper cites.

Square one bias in NLP: Towards a multi-dimensional exploration of the research manifold

Sebastian Ruder, Ivan Vulić, and Anders Søgaard. 2022 · 2022

Earlier work this paper cites.

Mistral 7b

Original

Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023 · 2023

Earlier work this paper cites.

Language models are multilingual chain-of-thought reasoners

Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, Dipanjan Das, and Jason Wei. 2023 · 2023

Cited alongside, same era.

Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback

Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher Manning. 2023 · 2023

Cited alongside, same era.

On the calibration of multilingual question answering llms

Original

Yahan Yang, Soham Dan, Dan Roth, and Insup Lee. 2023 · 2023

Cited alongside, same era.

Phi-4 technical report

Original

Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J. Hewett, Mojan Javaheripi, Piero Kauffmann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio C. T. Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Olli Saarikivi, and 8 others. 2024 · 2024

Cited alongside, same era.

Calibrating the confidence of large language models by eliciting fidelity

Mozhi Zhang, Mianqiu Huang, Rundong Shi, Linsen Guo, Chong Peng, Peng Yan, Yaqian Zhou, and Xipeng Qiu. 2024c · 2024

Later among the works it cites.

Layer swapping for zero-shot cross-lingual transfer in large language models

Lucas Bandarkar, Benjamin Muller, Pritish Yuvraj, Rui Hou, Nayan Singhal, Hongjiang Lv, and Bing Liu. 2025 · 2025

Closest in time.

The harms of class imbalance corrections for machine learning based prediction models: A simulation study

Alex Carriero, Kim Luijken, Anne de Hond, Karel G. M. Moons, Ben van Calster, and Maarten van Smeden. 2025 · 2025

Closest in time.

Mind the confidence gap: Overconfidence, calibration, and distractor effects in large language models

Original

Prateek Chhikara. 2025 · 2025

Closest in time.

Common Crawl

Common Crawl Foundation. 2025 · 2025

Closest in time.