2024

Calibration and Correctness of Language Models for Code

Spiess, Claudio, Gros, David, Pai, Kunal Suresh et al.

Understand

Machine learning models are widely used, but can also often be wrong.

Users would benefit from a reliable indication of whether a given output from a given model should be trusted, so a rational decision can be made whether to use the output or not.
For example, outputs can be associated with a confidence measure; if this confidence measure is strongly associated with likelihood of correctness, then the model is said to be well-calibrated.
A well-calibrated confidence measure can serve as a basis for rational, graduated decision-making on how much review and care is needed when using generated code.

Built on

“Language Models Are Few-Shot Learners”
Tom Brown et al · 1901
Earlier work this paper cites.
“CodeSearchNet Challenge: Evaluating the State of Semantic Code Search”, 2020
Original
Hamel Husain et al · 1909
Earlier work this paper cites.
“Verification of Forecasts Expressed in Terms of Probability”
Glenn. Brier · 1950
Earlier work this paper cites.
“Probabilistic Outputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods”
John Platt · 1999
Earlier work this paper cites.
“Outcome Prediction Model for Very Elderly Critically Ill Patients”
David. Nierman, Clyde. Schechter, Lisa. Cannon and Diane. Meier · 2001
Earlier work this paper cites.
“Obtaining Calibrated Probability Estimates from Decision Trees and Naive Bayesian Classifiers”
Bianca Zadrozny and Charles Elkan · 2001
Earlier work this paper cites.
“Transforming Classifier Scores into Accurate Multiclass Probability Estimates”
Bianca Zadrozny and Charles Elkan · 2002
Earlier work this paper cites.
“Assessing the Performance of Prediction Models: A Framework for Some Traditional and Novel Measures”
Ewout. Steyerberg et al · 2010
Earlier work this paper cites.
“Mining Source Code Repositories at Massive Scale Using Language Modeling”
Miltiadis Allamanis and Charles Sutton · 2013
Earlier work this paper cites.
“An Empirical Analysis of Flaky Tests”
Qingzhou Luo, Farah Hariri, Lamyaa Eloussi and Darko Marinov · 2014
Earlier work this paper cites.
“Defects4J: A Database of Existing Faults to Enable Controlled Testing Studies for Java Programs”
René Just, Darioush Jalali and Michael. Ernst · 2014
Earlier work this paper cites.
“Obtaining Well Calibrated Probabilities Using Bayesian Binning”
Mahdi Naeini, Gregory Cooper and Milos Hauskrecht · 2015
Earlier work this paper cites.
“Probabilistic Model for Code with Decision Trees”
Veselin Raychev, Pavol Bielik and Martin Vechev · 2016
Earlier work this paper cites.
“On Calibration of Modern Neural Networks”
Chuan Guo, Geoff Pleiss, Yu Sun and Kilian. Weinberger · 2017
Earlier work this paper cites.
“Mixup: Beyond Empirical Risk Minimization”, 2018
Hongyi Zhang, Moustapha Cisse, Yann. Dauphin and David Lopez-Paz · 2018
Earlier work this paper cites.
“GUESS: Projecting Machine Learning Scores to Well-Calibrated Probability Estimates for Clinical Decision-Making”
Johanna Schwarz and Dominik Heider · 2019
Earlier work this paper cites.
“Measuring Calibration in Deep Learning”, 2019, pp. 38–41
Jeremy Nixon et al · 2019
Earlier work this paper cites.
“Beyond Temperature Scaling: Obtaining Well-Calibrated Multi-Class Probabilities with Dirichlet Calibration”
Meelis Kull et al · 2019
Earlier work this paper cites.
“Automated Program Repair”
Claire Goues, Michael Pradel and Abhik Roychoudhury · 2019
Earlier work this paper cites.
“Using Pre-Training Can Improve Model Robustness and Uncertainty”
Dan Hendrycks, Kimin Lee and Mantas Mazeika · 2019
Earlier work this paper cites.
“Calibration of Pre-trained Transformers”
Shrey Desai and Greg Durrett · 2020
Earlier work this paper cites.
“How Often Do Single-Statement Bugs Occur? The ManySStuBs4J Dataset”
Rafael-Michael Karampatsis and Charles Sutton · 2020
Earlier work this paper cites.

Similar

“Operational Calibration: Debugging Confidence Errors for DNNs in the Field”
Zenan Li et al · 2020
Cited alongside, same era.
“You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion”, 2021, pp. 1559–1575
Roei Schuster, Congzheng Song, Eran Tromer and Vitaly Shmatikov · 2021
Cited alongside, same era.
“How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering”
Zhengbao Jiang, Jun Araki, Haibo Ding and Graham Neubig · 2021
Cited alongside, same era.
“Evaluating Large Language Models Trained on Code”, 2021
Original
Mark Chen et al · 2021
Cited alongside, same era.
“Program Synthesis with Large Language Models”, 2021
Original
Jacob Austin et al · 2021
Cited alongside, same era.
“Code Prediction by Feeding Trees to Transformers”
Seohyun Kim, Jinman Zhao, Yuchi Tian and Satish Chandra · 2021
Cited alongside, same era.
“CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation”
Shuai Lu et al · 2021
Cited alongside, same era.
“Revisiting the Calibration of Modern Neural Networks”
Matthias Minderer et al · 2021
Cited alongside, same era.
“Teaching Models to Express Their Uncertainty in Words”
Stephanie Lin, Jacob Hilton and Owain Evans · 2022
Cited alongside, same era.
“Language Models (Mostly) Know What They Know”, 2022
Original
Saurav Kadavath et al · 2022
Cited alongside, same era.
“CodeFill: Multi-Token Code Completion by Jointly Learning from Structure and Naming Sequences”
Maliheh Izadi, Roberta Gismondi and Georgios Gousios · 2022
Cited alongside, same era.
“Productivity Assessment of Neural Code Completion”
Albert Ziegler et al · 2022
Cited alongside, same era.

Then

“Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Models”
Kaitlyn Zhou, Dan Jurafsky and Tatsunori Hashimoto · 2023
Later among the works it cites.
“Just Ask for Calibration”
Katherine Tian et al · 2023
Later among the works it cites.
“Impact of Code Language Models on Automated Program Repair”
Nan Jiang, Kevin Liu, Thibaud Lutellier and Lin Tan · 2023
Later among the works it cites.
“Automated Repair of Programs from Large Language Models”
Zhiyu Fan et al · 2023
Later among the works it cites.
“Better Patching Using LLM Prompting, via Self-Consistency”
Toufique Ahmed and Premkumar Devanbu · 2023
Later among the works it cites.
“CodeGen2: Lessons for Training LLMs on Programming and Natural Languages”, 2023
Original
Erik Nijkamp et al · 2023
Later among the works it cites.
“Grounded Copilot: How Programmers Interact with Code-Generating Models”
Shraddha Barke, Michael. James and Nadia Polikarpova · 2023
Later among the works it cites.
“A Close Look into the Calibration of Pre-trained Language Models”
Yangyi Chen et al · 2023
Later among the works it cites.
“Holistic Evaluation of Language Models”
Rishi Bommasani, Percy Liang and Tony Lee · 2023
Later among the works it cites.
“Reflexion: Language Agents with Verbal Reinforcement Learning”, 2023
Original
Noah Shinn et al · 2023
Later among the works it cites.
“LitCab: Lightweight Language Model Calibration over Short- and Long-form Responses”
Xin Liu · 2023
Later among the works it cites.
“A Survey of Learning-based Automated Program Repair”
Quanjun Zhang et al · 2023
Later among the works it cites.
“Beyond the Imitation Game: Quantifying and Extrapolating the Capabilities of Language Models”, 2023
Original
Aarohi Srivastava et al · 2023
Later among the works it cites.
“Toward Trustworthy Neural Program Synthesis”, 2023
Original
Darren Key, Wen-Ding Li and Kevin Ellis · 2023
Later among the works it cites.
“R-U-SURE? uncertainty-aware code suggestions by maximizing utility across random user intents”
Daniel. Johnson, Daniel Tarlow and Christian Walder · 2023
Later among the works it cites.
“Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Models”, 2023
Original
Yuheng Huang et al · 2023
Later among the works it cites.
“LEVER: learning to verify language-to-code generation with execution”
Ansong Ni et al · 2023
Later among the works it cites.
“PACE-LM: Prompting and Augmentation for Calibrated Confidence Estimation with GPT-4 in Cloud Incident Root Cause Analysis”, 2023
Original
Dylan Zhang et al · 2023
Later among the works it cites.
“On Calibration of Pre-trained Code Models”
Zhenhao Zhou, Chaofeng Sha and Xin Peng · 2024
Closest in time.
“DyPyBench: A Benchmark of Executable Python Software”
Islem Bouzenia, Bajaj Krishan and Michael Pradel · 2024
Closest in time.
“A Survey of Large Language Models for Code: Evolution, Benchmarking, and Future Trends”, 2024
Original
Zibin Zheng et al · 2024
Closest in time.
“Enhancing Language Model Factuality via Activation-Based Confidence Calibration and Guided Decoding”
Original
Xin Liu · 2024
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…