2019

How Often Do Single-Statement Bugs Occur? The ManySStuBs4J Dataset

Karampatsis, Rafael-Michael, Sutton, Charles

Understand

Program repair is an important but difficult software engineering problem.

One way to achieve acceptable performance is to focus on classes of simple bugs, such as bugs with single statement fixes, or that match a small set of bug templates.
However, it is very difficult to estimate the recall of repair techniques for simple bugs, as there are no datasets about how often the associated bugs occur in code.
To fill this gap, we provide a dataset of 153,652 single statement bug-fix changes mined from 1,000 popular open-source Java projects, annotated by whether they match any of a set of 16 bug templates, inspired by state-of-the-art program repair techniques.

Built on

SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repair
Original
Zimin Chen, Steve Kommrusch, Michele Tufano, Louis-Noël Pouchet, Denys Poshyvanyk, and Martin Monperrus. 2019 · 1901
Earlier work this paper cites.
The Role of Debugging Within Software Engineering Environments
Monika A. F. Müllerburg. 1983 · 1983
Earlier work this paper cites.
Big Code != Big Vocabulary: Open-Vocabulary Models for Source Code
Rafael-Michael Karampatsis, Hlib Babii, Romain Robbes, Charles Sutton, and Andrea Janes. 2020 · 2003
Earlier work this paper cites.
When do changes induce fixes?. In International Workshop on Mining Software Repositories . ACM
Jacek Sliwerski, Thomas Zimmermann, and Andreas Zeller. 2005 · 2005
Earlier work this paper cites.
Automatic Identification of Bug-Introducing Changes. In 21st IEEE/ACM International Conference on Automated Software Engineering (ASE’06) . 81–90
S. Kim, T. Zimmermann, K. Pan, and E. J. Jr. Whitehead. 2006 · 2006
Earlier work this paper cites.
SZZ Revisited: Verifying when Changes Induce Fixes. In Proceedings of the 2008 Workshop on Defects in Large Software Systems (Seattle, Washington) (DEFECTS ’08) . ACM, New York, NY, USA, 32–36
Chadd Williams and Jaime Spacco. 2008 · 2008
Earlier work this paper cites.
Apache Maven
Frederic P. Miller, Agnes F. Vandome, and John McBrewster. 2010 · 2010
Earlier work this paper cites.
GenProg: A Generic Method for Automatic Software Repair
Claire Le Goues, ThanhVu Nguyen, Stephanie Forrest, and Westley Weimer. 2012 · 2011
Earlier work this paper cites.
The GHTorrent dataset and tool suite. In Proceedings of the 10th Working Conference on Mining Software Repositories (San Francisco, CA, USA) (MSR ’13) . IEEE Press, Piscataway, NJ, USA, 233–236
Georgios Gousios. 2013 · 2013
Earlier work this paper cites.

Similar

The impact of tangled code changes. In Working Conference on Mining Software Repositories . IEEE Press, 121–130
Kim Herzig and Andreas Zeller. 2013 · 2013
Cited alongside, same era.
Defects4J: A Database of Existing Faults to Enable Controlled Testing Studies for Java Programs. In Proceedings of the 2014 International Symposium on Software Testing and Analysis (San Jose, CA, USA) (ISSTA 2014) . ACM, New York, NY, USA, 437–440
René Just, Darioush Jalali, and Michael D. Ernst. 2014 · 2014
Cited alongside, same era.
Suggesting Accurate Method and Class Names. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (Bergamo, Italy) (ESEC/FSE 2015) . ACM, New York, NY, USA, 38–49
Miltiadis Allamanis, Earl T. Barr, Christian Bird, and Charles Sutton. 2015 · 2015
Cited alongside, same era.
The ManyBugs and IntroClass Benchmarks for Automated Repair of C Programs
C. Le Goues, N. Holtschulte, E. K. Smith, Y. Brun, P. Devanbu, S. Forrest, and W. Weimer. 2015 · 2015
Cited alongside, same era.
First, Debug the Test Oracle
X. Guo, M. Zhou, X. Song, M. Gu, and J. Sun. 2015 · 2015
Cited alongside, same era.
Staged Program Repair with Condition Synthesis. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (Bergamo, Italy) (ESEC/FSE 2015) . ACM, New York, NY, USA, 166–178
Fan Long and Martin Rinard. 2015 · 2015
Cited alongside, same era.
Tricorder: Building a Program Analysis Ecosystem. In International Conference on Software Engineering (ICSE)
Caitlin Sadowski, Jeffrey van Gogh, Ciera Jaspan, Emma Soederberg, and Collin Winter. 2015 · 2015
Cited alongside, same era.
Automatic Patch Generation by Learning Correct Code
Fan Long and Martin Rinard. 2016 · 2016
Cited alongside, same era.
On the" Naturalness" of Buggy Code (ICSE ’16) . ACM
Baishakhi Ray, Vincent Hellendoorn, Zhaopeng Tu, Connie Nguyen, Saheel Godhane, Alberto Bacchelli, and Premkumar Devanbu. 2016 · 2016
Cited alongside, same era.

Then

The CodRep Machine Learning on Source Code Competition
Original
Zimin Chen and Martin Monperrus. 2018 · 2018
Later among the works it cites.
How Many of All Bugs Do We Find? A Study of Static Bug Detectors. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering (Montpellier, France) (ASE 2018) . Association for Computing Machinery, New York, NY, USA, 317–328
Andrew Habib and Michael Pradel. 2018 · 2018
Later among the works it cites.
Automatic Software Repair: A Bibliography
Martin Monperrus. 2018 · 2018
Later among the works it cites.
DeepBugs: A Learning Approach to Name-based Bug Detection
Michael Pradel and Koushik Sen. 2018 · 2018
Later among the works it cites.
Bugs.Jar: A Large-scale, Diverse Dataset of Real-world Java Bugs. In Proceedings of the 15th International Conference on Mining Software Repositories (Gothenburg, Sweden) (MSR ’18) . ACM, New York, NY, USA, 10–13
Ripon K. Saha, Yingjun Lyu, Wing Lam, Hiroaki Yoshida, and Mukul R. Prasad. 2018 · 2018
Later among the works it cites.
An Empirical Investigation into Learning Bug-fixing Patches in the Wild via Neural Machine Translation. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering (Montpellier, France) (ASE 2018) . ACM, New York, NY, USA, 832–837
Michele Tufano, Cody Watson, Gabriele Bavota, Massimiliano Di Penta, Martin White, and Denys Poshyvanyk. 2018a · 2018
Later among the works it cites.
An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation
Original
Michele Tufano, Cody Watson, Gabriele Bavota, Massimiliano Di Penta, Martin White, and Denys Poshyvanyk. 2018b · 2018
Later among the works it cites.
Does Unit-Tested Code Crash? A Case Study of Eclipse
Original
Efstathia Chioteli, Ioannis Batas, and Diomidis Spinellis. 2019 · 2019
Closest in time.
A Convolutional Attention Network for Extreme Summarization of Source Code. In Proceedings of ICML 2016 , Vol. 48. 2091–2100
Miltiadis Allamanis, Hao Peng, and Charles A. Sutton. 2016 · 2091
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…