2022

CodeDSI: Differentiable Code Search

Nadeem, Usama, Ziems, Noah, Wu, Shaoen

Understand

Reimplementing solutions to previously solved software engineering problems is not only inefficient but also introduces inadequate and error-prone code.

Many existing methods achieve impressive performance on this issue by using autoregressive text-generation models trained on code.
However, these methods are not without their flaws.
The generated code from these models can be buggy, lack documentation, and introduce vulnerabilities that may go unnoticed by developers.

Built on

Dehghani, M., Zamani, H., Severyn, A., Kamps, J., and Croft, W. B. Neural ranking models with weak supervision. In Proceedings of the 40th International
2017
Earlier work this paper cites.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4- 9, 2017, Long Beach, CA, USA
2017
Earlier work this paper cites.
Gillick, D., Presta, A., and Tomar, G. S. End-to-end retrieval in continuous space. arXiv preprint arXiv: 1811.08008
2018
Earlier work this paper cites.
Husain, H., Wu, H., Gazit, T., Allamanis, M., Brockschmidt, M. Codesearchnet challenge: Evaluating the state of semantic code search. arXiv preprint arXiv: 1909.09436
2019
Earlier work this paper cites.
Kwiatkowski, T., Palomaki, J., Rhinehart, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Kelcey, M., Devlin, J., Lee, K., Toutanova, K. N., Jones, L., Chang, M., Dai, A., Uszkoreit, J., Le, Q., Petrov, S. Natural questions: a benchmark for question answering research. In Transactions of the Association of Computational Linguistics,
2019
Earlier work this paper cites.
Nogueira, R., Yang, W., Lin, J., and Cho, K. Document expansion by query prediction. arXiv preprint arXiv: 1904.08375
2019
Earlier work this paper cites.
De Cao, N., Izacard, G., Riedel, S., Petroni, F. Autoregressive entity retrieval. arXiv preprint arXiv: 2010.00904
2020
Earlier work this paper cites.

Similar

Feng, Z., Guo, D., et al. Codebert: A pre-trained model for programming and natural languages. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, EMNLP 2020, Online Event, 16-20 November 2020
2020
Cited alongside, same era.
Guu, K., Lee, K., Tung, Z., Pasupat, P., Chang, M. REALM: Retrieval-Augmented Language Model Pre-Training. In Proceedings of ICML 2020
2020
Cited alongside, same era.
Lachaux, M., Roziere, B., Chanussot, L., Lample, G. Unsupervised Translation of Programming Languages. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual
2020
Cited alongside, same era.
Roberts, A., Raffel, C., and Shazeer, N. How much knowledge can you pack into the parameters of a language model? arXiv preprint arXiv: 2002.08910
2020
Cited alongside, same era.
Borgeaud, S., Mensch, A., Hoffmann, J., Cai, T., Rutherford, E., Millican, K., Driessche, G. v. d., Lespiau, J.-B., Damoc, B., Clark, A., et al. Improving language models by retrieving from trillions of tokens. arXiv preprint arXiv: 2112.04426
2021
Cited alongside, same era.
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. O., Kaplan, J., et al. Evaluating Large Language Models Trained on Code. arXiv preprint arXiv: 2107.03374
2021
Cited alongside, same era.
Das, B., Chakraborty, S. An Improved Text Sentiment Classification Model Using TF-IDF and Next Word Negation. arXiv preprint arXiv: 1806.06407
Cited in the paper.

Then

Guo, D., Ren, S., Lu, S., Feng, Z., et al. Graphcodebert: Pre-training code representations with data flow. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021
2021
Later among the works it cites.
Lu, S., Guo, D., Ren, S., Huang, J., et al. CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation. arXiv preprint arXiv: 2102.04664
2021
Later among the works it cites.
Ni, J., Abrego, G. H., Constant, N., Ma, J., Hall, K. B., Cer, D., and Yang, Y. Sentence-t5: Scalable sentence encoders from pre-trained text-to-text models. arXiv preprint arXiv: 2108.08877
2021
Later among the works it cites.
Wang, Y., Wang, W., Joty, S., Hoi, S. C. H. CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: Association for Computational Linguistics 2021, November, Online and Punta Cana, Dominican Republic
2021
Later among the works it cites.
Gu, J., Chen, Z., Monperrus, M.,. Multimodal Representation for Neural Code Search. arXiv preprint arXiv: 2107.00992
2022
Closest in time.
Tay, Y., Tran, V. Q., et al. Transformer Memory as a Differentiable Search Index. arXiv preprint arXiv: 2202.06991
2022
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…