2020

Sequence Model Design for Code Completion in the Modern IDE

Aye, Gareth Ari, Kaiser, Gail E.

Understand

Code completion plays a prominent role in modern integrated development environments (IDEs).

Machine learning has become ubiquitous in analogous natural language writing and search software, surfacing more relevant autocompletions and search suggestions in fewer keystrokes.
Prior research has reported training high-accuracy, deep neural networks for modeling source code, but little attention has been given to the practical constraints imposed by interactive developer tools.
In particular, neural language models for source code modeling like the one described in Maybe Deep Neural Networks are the Best Choice for Modeling Source Code are framed around code completion, but only report accuracy of next-token prediction.

#1Maybe Deep Neural Networks are the Best Choice for Modeling Source Code
Original
#2 #2 · 1903
Earlier work this paper cites.
#1Graph-based Statistical Language Model for Code. In #2 (#2) . #2, #2, #2
#2 #2 · 1934
Earlier work this paper cites.
#1Response Time in Man-computer Conversational Transactions. In #2 (#2) . #2, #2, #2
#2 #2 · 1968
Earlier work this paper cites.
#1Speech understanding systems
#2 · 1977
Earlier work this paper cites.
#1A method for solving the convex programming problem with convergence rate $O (1/k^2)$
#2 #2 · 1983
Earlier work this paper cites.
#1Long Short-Term Memory
#2 #2 · 1997
Earlier work this paper cites.
#1Quick Training of Probabilistic Neural Nets by Importance Sampling
#2 #2 · 2003
Earlier work this paper cites.
#1Hierarchical probabilistic neural network language model
#2 #2 · 2005
Earlier work this paper cites.
#1How Are Java Software Developers Using the Eclipse IDE?
#2 #2 · 2006
Earlier work this paper cites.
#1Learning from Examples to Improve Code Completion Systems. In #2 (#2) . #2, #2, #2
#2 #2 · 2009
Earlier work this paper cites.
#1Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In #2. #2
#2 #2 · 2010
Earlier work this paper cites.
#1On the Naturalness of Software. In #2 (#2) . #2, #2, #2
#2 #2 · 2012
Earlier work this paper cites.

#1Improving neural networks by preventing co-adaptation of feature detectors
Original
#2 #2 · 2012
Cited alongside, same era.
#1Understanding the exploding gradient problem
Original
#2 #2 · 2012
Cited alongside, same era.
#1Mining Source Code Repositories at Massive Scale Using Language Modeling. In #2 (#2) . #2, #2, #2
#2 #2 · 2013
Cited alongside, same era.
#1A statistical semantic language model for source code. In #2
#2 #2 · 2013
Cited alongside, same era.
#1Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
Original
#2 #2 · 2014
Cited alongside, same era.
#1Code Completion with Statistical Language Models. In #2 (#2) . #2, #2, #2
#2 #2 · 2014
Cited alongside, same era.
#1Long Short-Term Memory Based Recurrent Neural Network Architectures for Large Vocabulary Speech Recognition
Original
#2 #2 · 2014
Cited alongside, same era.
#1Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Original
#2 #2 · 2015
Cited alongside, same era.
#1An empirical investigation of code completion usage by professional software developers
#2 #2 · 2015
Cited alongside, same era.
#1Neural Machine Translation of Rare Words with Subword Units
Original
#2 #2 · 2015
Cited alongside, same era.
#1Pointer networks. In #2. #2
#2 #2 · 2015
Cited alongside, same era.

#1Toward Deep Learning Software Repositories. In #2 (#2) . #2, #2, #2
#2 #2 · 2015
Later among the works it cites.
#1Learning Python Code Suggestion with a Sparse Pointer Network
Original
#2 #2 · 2016
Later among the works it cites.
#1PHOG: probabilistic model for code. In #2. #2
#2 #2 · 2016
Later among the works it cites.
#1Exploring the Limits of Language Modeling
Original
#2 #2 · 2016
Later among the works it cites.
#1Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
Original
#2 #2 · 2017
Later among the works it cites.
#1Code Completion with Neural Attention and Pointer Networks
Original
#2 #2 · 2017
Later among the works it cites.
#1DéJàVu: A Map of Code Duplicates on GitHub
#2 #2 · 2017
Later among the works it cites.
#1The Adverse Effects of Code Duplication in Machine Learning Models of Code
Original
#2 #2 · 2018
Later among the works it cites.
#1Understanding the Disharmony between Dropout and Batch Normalization by Variance Shift
Original
#2 #2 · 2018
Later among the works it cites.
#1On the State of the Art of Evaluation in Neural Language Models. In #2
#2 #2 · 2018
Later among the works it cites.
#1When Code Completion Fails: A Case Study on Real-world Completions. In #2 (#2) . #2, #2, #2
#2 #2 · 2019
Later among the works it cites.

alphaXiv searches the wider corpus for related work and actual follow-ups.

alphaXiv is searching for related work…

Reading the bibliography…