Fetching the paper…
Reading the bibliography…
In this paper, we propose shifting the focus of robustness evaluation for Neural Program Repair (NPR) techniques toward naturally-occurring data transformations.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 1901
Earlier work this paper cites.
Sequencer: Sequence-to-sequence learning for end-to-end program repair
Zimin Chen, Steve James Kommrusch, Michele Tufano, Louis-Noël Pouchet, Denys Poshyvanyk, and Martin Monperrus. 2021 · 1959
Earlier work this paper cites.
A coefficient of agreement for nominal scales
Jacob Cohen. 1960 · 1960
Earlier work this paper cites.
Snowball sampling
Leo A Goodman. 1961 · 1961
Earlier work this paper cites.
Measuring nominal scale agreement among many raters
Joseph L Fleiss. 1971 · 1971
Earlier work this paper cites.
PMD applied . Vol. 10
Tom Copeland. 2005 · 2005
Earlier work this paper cites.
Using thematic analysis in psychology
Virginia Braun and Victoria Clarke. 2006 · 2006
Earlier work this paper cites.
A reference collection for web spam. In ACM Sigir Forum , Vol. 40. ACM New York, NY, USA, 11–24
Carlos Castillo, Debora Donato, Luca Becchetti, Paolo Boldi, Stefano Leonardi, Massimo Santini, and Sebastiano Vigna. 2006 · 2006
Earlier work this paper cites.
Using static analysis to find bugs
Nathaniel Ayewah, William Pugh, David Hovemeyer, J David Morgenthaler, and John Penix. 2008 · 2008
Earlier work this paper cites.
Research methods for human-computer interaction . Vol. 10
Paul Cairns and Anna L Cox. 2008 · 2008
Earlier work this paper cites.
Selecting empirical methods for software engineering research
Steve Easterbrook, Janice Singer, Margaret-Anne Storey, and Daniela Damian. 2008 · 2008
Earlier work this paper cites.
Card sorting: Designing usable categories
Donna Spencer. 2009 · 2009
Earlier work this paper cites.
KenLM: Faster and smaller language model queries. In Proceedings of the sixth workshop on statistical machine translation . 187–197
Kenneth Heafield. 2011 · 2011
Earlier work this paper cites.
A general evaluation measure for document organization tasks. In Proceedings of the 36th International ACM SIGIR conference on Research and development in information retrieval . 643–652
Enrique Amigó, Julio Gonzalo, and Felisa Verdejo. 2013 · 2013
Earlier work this paper cites.
Automatic patch generation learned from human-written patches. In Proceedings of the 35th IEEE/ACM International Conference on Software Engineering (ICSE) . IEEE, 802–811
Dongsun Kim, Jaechang Nam, Jaewoo Song, and Sunghun Kim. 2013 · 2013
Earlier work this paper cites.
Syntax errors just aren’t natural: Improving error reporting with language models. In Proceedings of the 11th Working Conference on Mining Software Repositories . 252–261
Joshua Charles Campbell, Abram Hindle, and José Nelson Amaral. 2014 · 2014
Earlier work this paper cites.
Defects4J: A database of existing faults to enable controlled testing studies for Java programs. In Proceedings of the 23rd International Symposium on Software Testing and Analysis (ISSTA) . 437–440
René Just, Darioush Jalali, and Michael D Ernst. 2014 · 2014
Earlier work this paper cites.
Likert scale: Explored and explained
Ankur Joshi, Saket Kale, Satish Chandel, and D Kumar Pal. 2015 · 2015
Earlier work this paper cites.
The ManyBugs and IntroClass benchmarks for automated repair of C programs
Claire Le Goues, Neal Holtschulte, Edward K Smith, Yuriy Brun, Premkumar Devanbu, Stephanie Forrest, and Westley Weimer. 2015 · 2015
Earlier work this paper cites.
An analysis of patch plausibility and correctness for generate-and-validate patch generation systems. In Proceedings of the 2015 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) . Association for Computing Machinery, 24–36
Zichao Qi, Fan Long, Sara Achour, and Martin Rinard. 2015 · 2015
Earlier work this paper cites.
Is the cure worse than the disease? overfitting in automated program repair. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering . 532–543
Edward K Smith, Earl T Barr, Claire Le Goues, and Yuriy Brun. 2015 · 2015
Earlier work this paper cites.
On the naturalness of software
Abram Hindle, Earl T Barr, Mark Gabel, Zhendong Su, and Premkumar Devanbu. 2016 · 2016
Earlier work this paper cites.
History driven program repair. In Proceedings of the 23rd IEEE International Conference on Software Analysis, Evolution, and Reengineering (SANER) . IEEE, 213–224
Xuan Bach D Le, David Lo, and Claire Le Goues. 2016 · 2016
Earlier work this paper cites.
On the" naturalness" of buggy code. In Proceedings of the 38th International Conference on Software Engineering . IEEE, 428–439
Baishakhi Ray, Vincent Hellendoorn, Saheel Godhane, Zhaopeng Tu, Alberto Bacchelli, and Premkumar Devanbu. 2016 · 2016
Earlier work this paper cites.
Applying thematic analysis to define an awareness interpretation for collaborative computer games
Miguel A Teruel, Elena Navarro, Pascual González, Víctor López-Jaquero, and Francisco Montero. 2016 · 2016
Earlier work this paper cites.
S3: syntax-and semantic-guided repair synthesis via programming by examples. In Proceedings of the 11th Joint Meeting on Foundations of Software Engineering (ESEC/FSE) . Association for Computing Machinery, 593–604
Xuan-Bach D Le, Duc-Hiep Chu, David Lo, Claire Le Goues, and Willem Visser. 2017 · 2017
Earlier work this paper cites.
Codeflaws: a programming competition benchmark for evaluating automated program repair tools. In 2017 IEEE/ACM 39th International Conference on Software Engineering Companion (ICSE-C) . IEEE, 180–182
Shin Hwei Tan, Jooyong Yi, Sergey Mechtaev, Abhik Roychoudhury, et al · 2017
Earlier work this paper cites.
Bug characteristics in blockchain systems: a large-scale empirical study. In 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR) . IEEE, 413–424
Zhiyuan Wan, David Lo, Xin Xia, and Liang Cai. 2017 · 2017
Earlier work this paper cites.
AutoSpearman: Automatically mitigating correlated software metrics for interpreting defect models. In 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME) . IEEE Computer Society, 92–103
Jirayus Jiarpakdee, Chakkrit Tantithamthavorn, and Christoph Treude. 2018 · 2018
Earlier work this paper cites.
Are mutants really natural? a study on how" naturalness" helps mutant selection. In Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement . 1–10
Matthieu Jimenez, Thiery Titcheu Checkam, Maxime Cordy, Mike Papadakis, Marinos Kintis, Yves Le Traon, and Mark Harman. 2018 · 2018
Earlier work this paper cites.
Overfitting in semantics-based automated program repair. In Proceedings of the 40th International Conference on Software Engineering . 163–163
Xuan-Bach D Le, Ferdian Thung, David Lo, and Claire Le Goues. 2018 · 2018
Earlier work this paper cites.
Bugs. jar: A large-scale, diverse dataset of real-world java bugs. In Proceedings of the 15th international conference on mining software repositories . 10–13
Ripon K Saha, Yingjun Lyu, Wing Lam, Hiroaki Yoshida, and Mukul R Prasad. 2018 · 2018
Earlier work this paper cites.
An empirical investigation into learning bug-fixing patches in the wild via neural machine translation. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering . 832–837
Michele Tufano, Cody Watson, Gabriele Bavota, Massimiliano Di Penta, Martin White, and Denys Poshyvanyk. 2018 · 2018
Earlier work this paper cites.
Generating Natural Adversarial Examples. In International Conference on Learning Representations
Zhengli Zhao, Dheeru Dua, and Sameer Singh. 2018 · 2018
Earlier work this paper cites.
Bugsjs: a benchmark of javascript bugs. In 2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST) . IEEE, 90–101
Péter Gyimesi, Béla Vancsics, Andrea Stocco, Davood Mazinanian, Arpád Beszédes, Rudolf Ferenc, and Ali Mesbah. 2019 · 2019
Cited alongside, same era.
On reliability of patch correctness assessment. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) . IEEE, 524–535
Xuan-Bach D Le, Lingfeng Bao, David Lo, Xin Xia, Shanping Li, and Corina Pasareanu. 2019 · 2019
Cited alongside, same era.
You cannot fix what you cannot find! an investigation of fault localization bias in benchmarking automated program repair systems. In 2019 12th IEEE conference on software testing, validation and verification (ICST) . IEEE, 102–113
Kui Liu, Anil Koyuncu, Tegawendé F Bissyandé, Dongsun Kim, Jacques Klein, and Yves Le Traon. 2019a · 2019
Cited alongside, same era.
Bears: An extensible java bug benchmark for automatic program repair studies. In 2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER) . IEEE, 468–478
Fernanda Madeiral, Simon Urli, Marcelo Maia, and Martin Monperrus. 2019 · 2019
Codebert-nt: code naturalness via codebert. In 2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS) . IEEE, 936–947
Ahmed Khanfir, Matthieu Jimenez, Mike Papadakis, and Yves Le Traon. 2022 · 2022
Later among the works it cites.
Autopruner: transformer-based call graph pruning. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering . 520–532
Thanh Le-Cong, Hong Jin Kang, Truong Giang Nguyen, Stefanus Agus Haryono, David Lo, Xuan-Bach D Le, and Quyet Thang Huynh. 2022 · 2022
Later among the works it cites.
Dear: A novel deep learning-based approach for automated program repair. In Proceedings of the 44th International Conference on Software Engineering . 511–523
Yi Li, Shaohua Wang, and Tien N Nguyen. 2022 · 2022
Later among the works it cites.
FFL: Fine-grained Fault Localization for Student Programs via Syntactic and Semantic Reasoning. In 2022 IEEE International Conference on Software Maintenance and Evolution (ICSME) . IEEE, 151–162
Thanh-Dat Nguyen, Thanh Le-Cong, Duc-Minh Luong, Van-Hai Duong, Xuan-Bach D Le, David Lo, and Quyet-Thang Huynh. 2022b · 2022
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Natural software revisited. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) . IEEE, 37–48
Musfiqur Rahman, Dharani Palani, and Peter C Rigby. 2019 · 2019
Cited alongside, same era.
An empirical study on learning bug-fixing patches in the wild via neural machine translation
Michele Tufano, Cody Watson, Gabriele Bavota, Massimiliano Di Penta, Martin White, and Denys Poshyvanyk. 2019 · 2019
Cited alongside, same era.
How does machine learning change software development practices?
Zhiyuan Wan, Xin Xia, David Lo, and Gail C Murphy. 2019 · 2019
Cited alongside, same era.
Codit: Code editing with tree-based neural models
Saikat Chakraborty, Yangruibo Ding, Miltiadis Allamanis, and Baishakhi Ray. 2020 · 2020
Cited alongside, same era.
Patching as translation: the data and the metaphor. In Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering . IEEE, 275–286
Yangruibo Ding, Baishakhi Ray, Premkumar Devanbu, and Vincent J Hellendoorn. 2020 · 2020
Cited alongside, same era.
CodeBERT: A Pre-Trained Model for Programming and Natural Languages. In Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020 (Findings of ACL, Vol. EMNLP 2020) , Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, 1536–1547
Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. 2020 · 2020
Cited alongside, same era.
The impact of automated feature selection techniques on the interpretation of defect models
Jirayus Jiarpakdee, Chakkrit Tantithamthavorn, and Christoph Treude. 2020 · 2020
Cited alongside, same era.
DLfix: Context-based code transformation learning for automated program repair. In Proceedings of the 42nd IEEE/ACM International Conference on Software Engineering (ICSE) . IEEE, 602–614
Yi Li, Shaohua Wang, and Tien N Nguyen. 2020 · 2020
Cited alongside, same era.
Later among the works it cites.
Trust enhancement issues in program repair. In Proceedings of the 44th International Conference on Software Engineering . 2228–2240
Yannic Noller, Ridwan Shariffdeen, Xiang Gao, and Abhik Roychoudhury. 2022 · 2022
Later among the works it cites.
Bloom: A 176b-parameter open-access multilingual language model
Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al · 2022
Later among the works it cites.
Less training, more repairing please: revisiting automated program repair via zero-shot learning. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) . Association for Computing Machinery, 959–971
Chunqiu Steven Xia and Lingming Zhang. 2022 · 2022
Later among the works it cites.
Natural Attack for Pre-Trained Models of Code. In Proceedings of the 44th IEEE/ACM International Conference on Software Engineering (Pittsburgh, Pennsylvania) (ICSE ’22) . Association for Computing Machinery, New York, NY, USA, 1482–1493
Zhou Yang, Jieke Shi, Junda He, and David Lo. 2022 · 2022
Later among the works it cites.
SelfAPR: Self-supervised Program Repair with Test Execution Diagnostics. In 37th IEEE/ACM International Conference on Automated Software Engineering, ASE 2022, Rochester, MI, USA, October 10-14, 2022 . ACM, 92:1–92:13
He Ye, Matias Martinez, Xiapu Luo, Tao Zhang, and Martin Monperrus. 2022b · 2022
Later among the works it cites.
Data augmentation by program transformation
Shiwen Yu, Ting Wang, and Ji Wang. 2022 · 2022
Later among the works it cites.
Towards robustness of deep program processing models—detection, estimation, and enhancement
Huangzhao Zhang, Zhiyi Fu, Ge Li, Lei Ma, Zhehao Zhao, Hua’an Yang, Yizhe Sun, Yang Liu, and Zhi Jin. 2022 · 2022
Later among the works it cites.
StandUp4NPR: Standardizing SetUp for Empirically Comparing Neural Program Repair Systems. In 37th IEEE/ACM International Conference on Automated Software Engineering, ASE 2022, Rochester, MI, USA, October 10-14, 2022 . ACM, 97:1–97:13
Wenkang Zhong, Hongliang Ge, Hongfei Ai, Chuanyi Li, Kui Liu, Jidong Ge, and Bin Luo. 2022a · 2022
Later among the works it cites.
Beware of the Unexpected: Bimodal Taint Analysis (ISSTA 2023) . Association for Computing Machinery, New York, NY, USA, 211–222
Yiu Wai Chow, Max Schäfer, and Michael Pradel. 2023 · 2023
Later among the works it cites.
An Extensive Study on Adversarial Attack against Pre-trained Models of Code
Xiaohu Du, Ming Wen, Zichao Wei, Shangwen Wang, and Hai Jin. 2023 · 2023
Later among the works it cites.
How do humans perceive adversarial text? A reality check on the validity and naturalness of word-based adversarial attacks. In The 61st Annual Meeting Of The Association For Computational Linguistics
Salijona Dyrmishi, Salah GHAMIZI, and Maxime Cordy. 2023 · 2023
Later among the works it cites.
Non-adversarial robustness of deep learning methods for computer vision. In 2023 10th International Conference on Electrical, Electronic and Computing Engineering (IcETRAN) . IEEE, 1–9
Gorana Gojić, Vladimir Vincan, Ognjen Kundačina, Dragiša Mišković, and Dinu Dragan. 2023 · 2023
Later among the works it cites.
Impact of code language models on automated program repair. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) . IEEE, 1430–1442
Nan Jiang, Kevin Liu, Thibaud Lutellier, and Lin Tan. 2023a · 2023
Later among the works it cites.
The Future Can’t Help Fix The Past: Assessing Program Repair In The Wild
Vinay Kabadi, Dezhen Kong, Siyu Xie, Lingfeng Bao, Gede Artha Azriadi Prana, Tien-Duy B Le, Xuan-Bach D Le, and David Lo. 2023 · 2023
Later among the works it cites.
Invalidator: Automated patch correctness assessment via semantic and syntactic reasoning
Thanh Le-Cong, Duc-Minh Luong, Xuan Bach D Le, David Lo, Nhat-Hoa Tran, Bui Quang-Huy, and Quyet-Thang Huynh. 2023a · 2023
Later among the works it cites.
Refining ChatGPT-generated code: Characterizing and mitigating code quality issues
Yue Liu, Thanh Le-Cong, Ratnadira Widyasari, Chakkrit Tantithamthavorn, Li Li, Xuan-Bach D Le, and David Lo. 2023 · 2023
Later among the works it cites.
Multi-Granularity Detector for Vulnerability Fixes
Truong Giang Nguyen, Thanh Le-Cong, Hong Jin Kang, Ratnadira Widyasari, Chengran Yang, Zhipeng Zhao, Bowen Xu, Jiayuan Zhou, Xin Xia, Ahmed E Hassan, et al · 2023
Later among the works it cites.
Code llama: Open foundation models for code
Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, et al · 2023
Later among the works it cites.
Cerberus: A Program Repair Framework. In Proceedings of the ACM/IEEE 45th International Conference on Software Engineering: Companion Proceedings (Melbourne, Australia) (ICSE ’23) . IEEE, 73–77
Ridwan Shariffdeen, Martin Mirchev, Yannic Noller, and Abhik Roychoudhury. 2023 · 2023
Later among the works it cites.
Repairllama: Efficient representations and fine-tuned adapters for program repair
André Silva, Sen Fang, and Martin Monperrus. 2023 · 2023
Later among the works it cites.
Adversarial Attacks on Neural Models of Code via Code Difference Reduction
Zhao Tian, Junjie Chen, and Zhi Jin. 2023 · 2023
Later among the works it cites.
Llama 2: Open foundation and fine-tuned chat models
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al · 2023
Later among the works it cites.
Natural robustness of machine learning in the open world
Hongxin Wei. 2023 · 2023
Later among the works it cites.
How do developers really feel about bug fixing? Directions for automatic program repair
Emily Winter, David Bowes, Steve Counsell, Tracy Hall, Sæmundur Haraldsson, Vesna Nowack, and John Woodward. 2023 · 2023
Later among the works it cites.
Challenging Machine Learning-Based Clone Detectors via Semantic-Preserving Code Transformations
Weiwei Zhang, Shengjian Guo, Hongyu Zhang, Yulei Sui, Yinxing Xue, and Yun Xu. 2023a · 2023
Later among the works it cites.
Data Augmentation Approaches for Source Code Models: A Survey
Terry Yue Zhuo, Zhou Yang, Zhensu Sun, Yufei Wang, Li Li, Xiaoning Du, Zhenchang Xing, and David Lo. 2023 · 2023
Later among the works it cites.
Fixminer: Mining relevant fix patterns for automated program repair
Anil Koyuncu, Kui Liu, Tegawendé F Bissyandé, Dongsun Kim, Jacques Klein, Martin Monperrus, and Yves Le Traon. 2020 · 2024
Closest in time.
Semantic-guided Search for Efficient Program Repair with Large Language Models
Thanh Le-Cong, Bach Le, and Toby Murray. 2024 · 2024
Closest in time.
Natural Symbolic Execution-Based Testing for Big Data Analytics
Yaoxuan Wu, Ahmad Humayun, Muhammad Ali Gulzar, and Miryung Kim. 2024 · 2024
Closest in time.