2019

Structural Language Models of Code

Alon, Uri, Sadaka, Roy, Levy, Omer et al.

Understand

We address the problem of any-code completion - generating a missing piece of source code in a given program without any restriction on the vocabulary or structure.

We introduce a new approach to any-code completion that leverages the strict syntax of programming languages to model a code snippet as a tree - structural language modeling (SLM).
SLM estimates the probability of the program's abstract syntax tree (AST) by decomposing it into a product of conditional probabilities over its nodes.
We present a neural model that computes these conditional probabilities by considering all AST paths leading to a target node.

Built on

Prow: A step toward automatic program writing
Waldinger, R. J. and Lee, R. C · 1969
Earlier work this paper cites.
Application of theorem proving to problem solving
Green, C · 1981
Earlier work this paper cites.
On the synthesis of a reactive module
Pnueli, A. and Rosner, R · 1989
Earlier work this paper cites.
Automating string processing in spreadsheets using input-output examples
Gulwani, S · 2011
Earlier work this paper cites.
Structured generative models of natural source code
Maddison, C. and Tarlow, D · 2014
Earlier work this paper cites.
Bimodal modelling of source code and natural language
Allamanis, M., Tarlow, D., Gordon, A., and Wei, Y · 2015
Earlier work this paper cites.
Adam: A method for stochastic optimization
Kingma, D. and Ba, J · 2015
Earlier work this paper cites.
Effective approaches to attention-based neural machine translation
Luong, M.-T., Pham, H., and Manning, C. D · 2015
Earlier work this paper cites.
Flashmeta: a framework for inductive program synthesis
Polozov, O. and Gulwani, S · 2015
Earlier work this paper cites.
A convolutional attention network for extreme summarization of source code
Allamanis, M., Peng, H., and Sutton, C · 2016
Earlier work this paper cites.
Phog: probabilistic model for code
Bielik, P., Raychev, V., and Vechev, M · 2016
Earlier work this paper cites.
Incorporating copying mechanism in sequence-to-sequence learning
Gu, J., Lu, Z., Li, H., and Li, V. O · 2016
Earlier work this paper cites.
Latent predictor networks for code generation
Ling, W., Blunsom, P., Grefenstette, E., Hermann, K. M., Kočiskỳ, T., Wang, F., and Senior, A · 2016
Earlier work this paper cites.
Learning programs from noisy data
Raychev, V., Bielik, P., Vechev, M., and Krause, A · 2016
Earlier work this paper cites.
Sequence-based structured prediction for semantic parsing
Xiao, C., Dymetman, M., and Gardent, C · 2016
Earlier work this paper cites.

Similar

Towards string-to-tree neural machine translation
Aharoni, R. and Goldberg, Y · 2017
Cited alongside, same era.
Neural attribute machines for program generation
Original
Amodio, M., Chaudhuri, S., and Reps, T · 2017
Cited alongside, same era.
Deepcoder: Learning to write programs
Balog, M., Gaunt, A. L., Brockschmidt, M., Nowozin, S., and Tarlow, D · 2017
Cited alongside, same era.
Robustfill: Neural program learning under noisy i/o
Devlin, J., Uesato, J., Bhupatiraju, S., Singh, R., Mohamed, A.-r., and Kohli, P · 2017
Cited alongside, same era.
Differentiable programs with neural libraries
Gaunt, A. L., Brockschmidt, M., Kushman, N., and Tarlow, D · 2017
Cited alongside, same era.
Program synthesis
Gulwani, S., Polozov, O., Singh, R., et al · 2017
Cited alongside, same era.
Opennmt: Open-source toolkit for neural machine translation
Klein, G., Kim, Y., Deng, Y., Senellart, J., and Rush, A. M · 2017
Cited alongside, same era.
Neuro-symbolic program synthesis
Parisotto, E., Mohamed, A.-r., Singh, R., Li, L., Zhou, D., and Kohli, P · 2017
Cited alongside, same era.
Abstract syntax networks for code generation and semantic parsing
Rabinovich, M., Stern, M., and Klein, D · 2017
Cited alongside, same era.
Attention is all you need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I · 2017
Cited alongside, same era.
A syntactic neural model for general-purpose code generation
Yin, P. and Neubig, G · 2017
Cited alongside, same era.
Learning to represent programs with graphs
Allamanis, M., Brockschmidt, M., and Khademi, M · 2018
Cited alongside, same era.

Then

Mapping language to code in programmatic context
Iyer, S., Konstas, I., Cheung, A., and Zettlemoyer, L · 2018
Later among the works it cites.
Neural sketch learning for conditional program generation
Murali, V., Qi, L., Chaudhuri, S., and Jermaine, C · 2018
Later among the works it cites.
Typesql: Knowledge-based type-aware neural text-to-sql generation
Yu, T., Li, Z., Zhang, Z., Zhang, R., and Radev, D · 2018
Later among the works it cites.
The adverse effects of code duplication in machine learning models of code
Allamanis, M · 2019
Closest in time.
Generative code modeling with graphs
Brockschmidt, M., Allamanis, M., Gaunt, A. L., and Polozov, O · 2019
Closest in time.
Write, execute, assess: Program synthesis with a repl
Ellis, K., Nye, M., Pu, Y., Sosa, F., Tenenbaum, J., and Solar-Lezama, A · 2019
Closest in time.
Structured neural summarization
Fernandes, P., Allamanis, M., and Brockschmidt, M · 2019
Closest in time.
Learning programmatic idioms for scalable semantic parsing
Iyer, S., Cheung, A., and Zettlemoyer, L · 2019
Closest in time.
Learning a meta-solver for syntax-guided program synthesis
Si, X., Yang, Y., Dai, H., Naik, M., and Song, L · 2019
Closest in time.
Learning to represent edits
Yin, P., Neubig, G., Allamanis, M., Brockschmidt, M., and Gaunt, A. L · 2019
Closest in time.
Learning neurosymbolic generative models via program synthesis
Young, H., Bastani, O., and Naik, M · 2019
Closest in time.
Neural networks for modeling source code edits
Original
Zhao, R., Bieber, D., Swersky, K., and Tarlow, D · 2019
Closest in time.
On the bottleneck of graph neural networks and its practical implications
Original
Alon, U. and Yahav, E · 2020
Closest in time.
Neural edit completion
Original
Brody, S., Alon, U., and Yahav, E · 2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…