2020

CC2Vec: Distributed Representations of Code Changes

Hoang, Thong, Kang, Hong Jin, Lawall, Julia et al.

Understand

Existing work on software patches often use features specific to a single task.

These works often rely on manually identified features, and human effort is required to identify these features for each task.
In this work, we propose CC2Vec, a neural network model that learns a representation of code changes guided by their accompanying log messages, which represent the semantic intent of the code changes.
CC2Vec models the hierarchical structure of a code change with the help of the attention mechanism and uses multiple comparison functions to identify the differences between the removed and added code.

Built on

Dropout: a simple way to prevent neural networks from overfitting
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014 · 1958
Earlier work this paper cites.
Training feedforward networks with the Marquardt algorithm
Martin T Hagan and Mohammad B Menhaj. 1994 · 1994
Earlier work this paper cites.
Introduction to multi-layer feed-forward neural networks
Daniel Svozil, Vladimir Kvasnicka, and Jiri Pospichal. 1997 · 1997
Earlier work this paper cites.
Svmlight: Support vector machine
Thorsten Joachims. 1999 · 1999
Earlier work this paper cites.
Predicting risk of software changes
Audris Mockus and David M Weiss. 2000 · 2000
Earlier work this paper cites.
Overfitting in neural nets: Backpropagation, conjugate gradient, and early stopping. In Advances in neural information processing systems . 402–408
Rich Caruana, Steve Lawrence, and C Lee Giles. 2001 · 2001
Earlier work this paper cites.
BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics . Association for Computational Linguistics, 311–318
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002 · 2002
Earlier work this paper cites.
Learning with positive and unlabeled examples using weighted logistic regression. In ICML , Vol. 3. 448–455
Wee Sun Lee and Bing Liu. 2003 · 2003
Earlier work this paper cites.
NLTK: the natural language toolkit. In Proceedings of the ACL 2004 on Interactive poster and demonstration sessions . Association for Computational Linguistics, 31
Steven Bird and Edward Loper. 2004 · 2004
Earlier work this paper cites.
Learning from bug-introducing changes to prevent fault prone code. In Ninth international workshop on Principles of software evolution: in conjunction with the 6th ESEC/FSE joint meeting . ACM, 19–26
Lerina Aversano, Luigi Cerulo, and Concettina Del Grosso. 2007 · 2007
Earlier work this paper cites.
Efficient bounds for the softmax function and applications to approximate inference in hybrid models. In NIPS 2007 workshop for approximate Bayesian inference in continuous/hybrid systems
Guillaume Bouchard. 2007 · 2007
Earlier work this paper cites.
Classifying software changes: Clean or buggy?
Sunghun Kim, E James Whitehead Jr, and Yi Zhang. 2008 · 2008
Earlier work this paper cites.
Semi-supervised learning (chapelle, o. et al., eds.; 2006)[book reviews]
Olivier Chapelle, Bernhard Scholkopf, and Alexander Zien. 2009 · 2009
Earlier work this paper cites.
Introduction to information retrieval
Christopher Manning, Prabhakar Raghavan, and Hinrich Schütze. 2010 · 2010
Earlier work this paper cites.
Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th international conference on machine learning (ICML-10) . 807–814
Vinod Nair and Geoffrey E Hinton. 2010 · 2010
Earlier work this paper cites.
Univariate hyperbolic tangent neural network approximation
George A Anastassiou. 2011 · 2011
Earlier work this paper cites.
Action recognition from a distributed representation of pose and appearance. In CVPR 2011 . IEEE, 3177–3184
Subhransu Maji, Lubomir Bourdev, and Jitendra Malik. 2011 · 2011
Earlier work this paper cites.
Convolutional neural networks applied to human face classification. In 2012 11th International Conference on Machine Learning and Applications , Vol. 2. IEEE, 580–583
Brian Cheung. 2012 · 2012
Earlier work this paper cites.
3D convolutional neural networks for human action recognition
Shuiwang Ji, Wei Xu, Ming Yang, and Kai Yu. 2012 · 2012
Earlier work this paper cites.
A large-scale empirical study of just-in-time quality assurance
Yasutaka Kamei, Emad Shihab, Bram Adams, Ahmed E Hassan, Audris Mockus, Anand Sinha, and Naoyasu Ubayashi. 2012 · 2012
Earlier work this paper cites.

Similar

An industrial study on the risk of software changes. In Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering . ACM, 62
Emad Shihab, Ahmed E Hassan, Bram Adams, and Zhen Ming Jiang. 2012 · 2012
Cited alongside, same era.
Identifying Linux bug fixing patches. In Proceedings of the 34th International Conference on Software Engineering . IEEE Press, 386–396
Yuan Tian, Julia Lawall, and David Lo. 2012 · 2012
Cited alongside, same era.
Improving deep neural networks for LVCSR using rectified linear units and dropout. In 2013 IEEE international conference on acoustics, speech and signal processing . IEEE, 8609–8613
George E Dahl, Tara N Sainath, and Geoffrey E Hinton. 2013 · 2013
Cited alongside, same era.
Boa: A language and infrastructure for analyzing ultra-large-scale software repositories. In Proceedings of the 2013 International Conference on Software Engineering . IEEE Press, 422–431
Robert Dyer, Hoan Anh Nguyen, Hridesh Rajan, and Tien N Nguyen. 2013 · 2013
Cited alongside, same era.
Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems . 3111–3119
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013 · 2013
Cited alongside, same era.
Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing . 1631–1642
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Ng, and Christopher Potts. 2013 · 2013
Cited alongside, same era.
Neural machine translation by jointly learning to align and translate
Original
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014 · 2014
Cited alongside, same era.
Convolutional Neural Networks for Sentence Classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) . Association for Computational Linguistics, Doha, Qatar, 1746–1751
Yoon Kim. 2014 · 2014
Cited alongside, same era.
Adam: A method for stochastic optimization
Original
Diederik P Kingma and Jimmy Ba. 2014 · 2014
Cited alongside, same era.
Hey! are you committing tangled changes?. In Proceedings of the 22nd International Conference on Program Comprehension . ACM, 262–265
Hiroyuki Kirinuki, Yoshiki Higo, Keisuke Hotta, and Shinji Kusumoto. 2014 · 2014
Cited alongside, same era.
Changescribe: A tool for automatically generating commit messages. In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering , Vol. 2. IEEE, 709–712
Mario Linares-Vásquez, Luis Fernando Cortés-Coy, Jairo Aponte, and Denys Poshyvanyk. 2015 · 2015
Cited alongside, same era.
TextRank based search term identification for software change tasks. In 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER) . IEEE, 540–544
Mohammad Masudur Rahman and Chanchal K Roy. 2015 · 2015
Cited alongside, same era.

Then

Are fix-inducing changes a moving target? a longitudinal case study of just-in-time defect prediction
Shane McIntosh and Yasutaka Kamei. 2017 · 2017
Later among the works it cites.
Exploring API embedding for API usages and applications. In 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE) . IEEE, 438–449
Trong Duc Nguyen, Anh Tuan Nguyen, Hung Dang Phan, and Tien N Nguyen. 2017 · 2017
Later among the works it cites.
Dynamic neural program embedding for program repair
Original
Ke Wang, Rishabh Singh, and Zhendong Su. 2017 · 2017
Later among the works it cites.
A Compare-Aggregate Model for Matching Text Sequences. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings
Shuohang Wang and Jing Jiang. 2017 · 2017
Later among the works it cites.
A survey of machine learning for big code and naturalness
Miltiadis Allamanis, Earl T Barr, Premkumar Devanbu, and Charles Sutton. 2018 · 2018
Later among the works it cites.
Deep attention neural tensor network for visual question answering. In Proceedings of the European Conference on Computer Vision (ECCV) . 20–35
Yalong Bai, Jianlong Fu, Tiejun Zhao, and Tao Mei. 2018 · 2018
Later among the works it cites.
Path-based function embedding and its application to error-handling specification mining. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering . ACM, 423–433
Daniel DeFreez, Aditya V Thakur, and Cindy Rubio-González. 2018 · 2018
Later among the works it cites.
Deep code search. In 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE) . IEEE, 933–944
Xiaodong Gu, Hongyu Zhang, and Sunghun Kim. 2018 · 2018
Later among the works it cites.
Code vectors: Understanding programs through embedded abstracted symbolic traces. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering . ACM, 163–174
Jordan Henkel, Shuvendu K Lahiri, Ben Liblit, and Thomas Reps. 2018 · 2018
Later among the works it cites.
Deep code comment generation. In Proceedings of the 26th Conference on Program Comprehension . ACM, 200–210
Xing Hu, Ge Li, Xin Xia, David Lo, and Zhi Jin. 2018 · 2018
Later among the works it cites.
Neural-machine-translation-based commit message generation: how far are we?. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering . ACM, 373–384
Zhongxin Liu, Xin Xia, Ahmed E Hassan, David Lo, Zhenchang Xing, and Xinyu Wang. 2018 · 2018
Later among the works it cites.
Learning to mine aligned code and natural language pairs from stack overflow. In 2018 IEEE/ACM 15th International Conference on Mining Software Repositories (MSR) . IEEE, 476–486
Pengcheng Yin, Bowen Deng, Edgar Chen, Bogdan Vasilescu, and Graham Neubig. 2018 · 2018
Later among the works it cites.
code2vec: Learning distributed representations of code
Uri Alon, Meital Zilberstein, Omer Levy, and Eran Yahav. 2019b · 2019
Later among the works it cites.
Semantic source code models using identifier embeddings. In Proceedings of the 16th International Conference on Mining Software Repositories . IEEE Press, 29–33
Vasiliki Efstathiou and Diomidis Spinellis. 2019 · 2019
Later among the works it cites.
PatchNet: Hierarchical Deep Learning-Based Stable Patch Identification for the Linux Kernel
Thong Hoang, Julia Lawall, Yuan Tian, Richard J Oentaryo, and David Lo. 2019b · 2019
Later among the works it cites.
PathMiner: a library for mining of path-based representations of code. In Proceedings of the 16th International Conference on Mining Software Repositories . IEEE Press, 13–17
Vladimir Kovalenko, Egor Bogomolov, Timofey Bryksin, and Alberto Bacchelli. 2019 · 2019
Later among the works it cites.
A neural model for generating natural language summaries of program subroutines. In Proceedings of the 41st International Conference on Software Engineering . IEEE Press, 795–806
Alexander LeClair, Siyuan Jiang, and Collin McMillan. 2019 · 2019
Later among the works it cites.
Import2vec learning embeddings for software libraries. In Proceedings of the 16th International Conference on Mining Software Repositories . IEEE Press, 18–28
Bart Theeten, Frederik Vandeputte, and Tom Van Cutsem. 2019 · 2019
Later among the works it cites.
Studying just-in-time defect prediction using cross-project models
Yasutaka Kamei, Takafumi Fukushima, Shane McIntosh, Kazuhiro Yamashita, Naoyasu Ubayashi, and Ahmed E Hassan. 2016 · 2072
Closest in time.
Summarizing source code using a neural attention model. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . 2073–2083
Srinivasan Iyer, Ioannis Konstas, Alvin Cheung, and Luke Zettlemoyer. 2016 · 2083
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…