Fetching the paper…
Reading the bibliography…
Recent breakthroughs in pre-trained code models, such as CodeBERT and Codex, have shown their superior performance in various downstream tasks.
K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, ‘‘Bleu: a method for automatic evaluation of machine translation,’’ in Proceedings of the 40th annual meeting of the Association for Computational Linguistics , 2002, pp. 311–318
2002
Earlier work this paper cites.
F. Thung, S. Wang, D. Lo, and J. Lawall, ‘‘Automatic recommendation of api methods from feature requests,’’ in 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE) . IEEE, 2013, pp. 290–300
2013
Earlier work this paper cites.
A. Hindle, E. T. Barr, M. Gabel, Z. Su, and P. Devanbu, ‘‘On the naturalness of software,’’ Communications of the ACM , vol. 59, no. 5, pp. 122–131, 2016
2016
Earlier work this paper cites.
M. M. Rahman, C. K. Roy, and D. Lo, ‘‘Rack: Automatic api recommendation using crowdsourced knowledge,’’ in 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER) , vol. 1. IEEE, 2016, pp. 349–359
2016
Earlier work this paper cites.
X. Gu, H. Zhang, D. Zhang, and S. Kim, ‘‘Deep api learning,’’ in Proceedings of the 2016 24th ACM SIGSOFT international symposium on foundations of software engineering , 2016, pp. 631–642
2016
Earlier work this paper cites.
2017
Earlier work this paper cites.
M. Allamanis, E. T. Barr, P. Devanbu, and C. Sutton, ‘‘A survey of machine learning for big code and naturalness,’’ ACM Computing Surveys (CSUR) , vol. 51, no. 4, pp. 1–37, 2018
2018
Earlier work this paper cites.
Q. Huang, X. Xia, Z. Xing, D. Lo, and X. Wang, ‘‘Api method recommendation without worrying about the task-api knowledge gap,’’ in Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering , 2018, pp. 293–304
2018
Earlier work this paper cites.
H. Li, S. Li, J. Sun, Z. Xing, X. Peng, M. Liu, and X. Zhao, ‘‘Improving api caveats accessibility by mining api caveats knowledge graph,’’ in 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME) . IEEE, 2018, pp. 183–193
2018
Earlier work this paper cites.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, ‘‘Bert: Pre-training of deep bidirectional transformers for language understanding,’’ in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , 2019, pp. 4171–4186
2019
Earlier work this paper cites.
2019
Earlier work this paper cites.
F. Petroni, T. Rocktäschel, S. Riedel, P. Lewis, A. Bakhtin, Y. Wu, and A. Miller, ‘‘Language models as knowledge bases?’’ in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , 2019, pp. 2463–2473
2019
Earlier work this paper cites.
Z. Feng, D. Guo, D. Tang, N. Duan, X. Feng, M. Gong, L. Shou, B. Qin, T. Liu, D. Jiang et al. , ‘‘Codebert: A pre-trained model for programming and natural languages,’’ in Findings of the Association for Computational Linguistics: EMNLP 2020 , 2020, pp. 1536–1547
2020
Earlier work this paper cites.
Z. Jiang, F. F. Xu, J. Araki, and G. Neubig, ‘‘How can we know what language models know?’’ Transactions of the Association for Computational Linguistics , vol. 8, pp. 423–438, 2020
2020
Earlier work this paper cites.
X. Qiu, T. Sun, Y. Xu, Y. Shao, N. Dai, and X. Huang, ‘‘Pre-trained models for natural language processing: A survey,’’ Science China Technological Sciences , vol. 63, no. 10, pp. 1872–1897, 2020
2020
Earlier work this paper cites.
M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer, ‘‘Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension,’’ in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , 2020, pp. 7871–7880
2020
Cited alongside, same era.
Z. Jiang, A. Anastasopoulos, J. Araki, H. Ding, and G. Neubig, ‘‘X-factr: Multilingual factual knowledge retrieval from pretrained language models,’’ in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , 2020, pp. 5943–5959
2020
Cited alongside, same era.
F. Petroni, P. Lewis, A. Piktus, T. Rocktäschel, Y. Wu, A. H. Miller, and S. Riedel, ‘‘How context affects language models’ factual predictions,’’ in Automated Knowledge Base Construction , 2020. [Online]. Available: https://openreview.net/forum?id=025X0zPfn
2020
Cited alongside, same era.
Y. Wan, W. Zhao, H. Zhang, Y. Sui, G. Xu, and H. Jin, ‘‘What do they capture? a structural analysis of pre-trained language models for source code,’’ in Proceedings of the 44th International Conference on Software Engineering , 2022, pp. 2377–2388
2022
Later among the works it cites.
J. A. Hernández López, M. Weyssow, J. S. Cuadrado, and H. Sahraoui, ‘‘Ast-probe: Recovering abstract syntax trees from hidden representations of pre-trained language models,’’ in 37th IEEE/ACM International Conference on Automated Software Engineering , 2022, pp. 1–11
2022
Later among the works it cites.
2022
Later among the works it cites.
Q. Huang, Z. Yuan, Z. Xing, X. Xu, L. Zhu, and Q. Lu, ‘‘Prompt-tuned code language model as a neural knowledge base for type inference in statically-typed partial code,’’ in 37th IEEE/ACM International Conference on Automated Software Engineering , 2022, pp. 1–13
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz et al. , ‘‘Transformers: State-of-the-art natural language processing,’’ in Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations , 2020, pp. 38–45
2020
Cited alongside, same era.
X. Ren, X. Ye, Z. Xing, X. Xia, X. Xu, L. Zhu, and J. Sun, ‘‘Api-misuse detection driven by fine-grained api-constraint knowledge graph,’’ in Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering , 2020, pp. 461–472
2020
Cited alongside, same era.
D. Guo, S. Ren, S. Lu, Z. Feng, D. Tang, S. LIU, L. Zhou, N. Duan, A. Svyatkovskiy, S. Fu, M. Tufano, S. K. Deng, C. Clement, D. Drain, N. Sundaresan, J. Yin, D. Jiang, and M. Zhou, ‘‘Graphcode{bert}: Pre-training code representations with data flow,’’ in International Conference on Learning Representations , 2021. [Online]. Available: https://openreview.net/forum?id=jLoC4ez43PZ
2021
Cited alongside, same era.
2021
Cited alongside, same era.
W. Ahmad, S. Chakraborty, B. Ray, and K.-W. Chang, ‘‘Unified pre-training for program understanding and generation,’’ in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , 2021, pp. 2655–2668
2021
Cited alongside, same era.
E. Perez, D. Kiela, and K. Cho, ‘‘True few-shot learning with language models,’’ Advances in neural information processing systems , vol. 34, pp. 11 054–11 070, 2021
2021
Cited alongside, same era.
G. Qin and J. Eisner, ‘‘Learning how to ask: Querying lms with mixtures of soft prompts,’’ in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , 2021, pp. 5203–5212
2021
Cited alongside, same era.
J. Wang, L. Li, and A. Zeller, ‘‘Restoring execution environments of jupyter notebooks,’’ in 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) . IEEE, 2021, pp. 1622–1633
2021
Cited alongside, same era.
2021
Cited alongside, same era.
2022
Later among the works it cites.
J. Martin and J. L. Guo, ‘‘Deep api learning revisited,’’ in Proceedings of the 30th IEEE/ACM International Conference on Program Comprehension , 2022, pp. 321–330
2022
Later among the works it cites.
M. A. Hadi, I. N. B. Yusuf, F. Thung, K. G. Luong, J. Lingxiao, F. H. Fard, and D. Lo, ‘‘On the effectiveness of pretrained models for api learning,’’ in Proceedings of the 30th IEEE/ACM International Conference on Program Comprehension , 2022, pp. 309–320
2022
Later among the works it cites.
X. Liu, K. Ji, Y. Fu, W. Tam, Z. Du, Z. Yang, and J. Tang, ‘‘P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks,’’ in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , 2022, pp. 61–68
2022
Later among the works it cites.
2023
Closest in time.
OpenAI, ‘‘Gpt-4 technical report,’’ 2023
2023
Closest in time.
2023
Closest in time.
P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, and G. Neubig, ‘‘Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing,’’ ACM Computing Surveys , vol. 55, no. 9, pp. 1–35, 2023
2023
Closest in time.
2023
Closest in time.
N. Carlini, D. Ippolito, M. Jagielski, K. Lee, F. Tramer, and C. Zhang, ‘‘Quantifying memorization across neural language models,’’ in The Eleventh International Conference on Learning Representations , 2023. [Online]. Available: https://openreview.net/forum?id=TatRHT_1cK
2023
Closest in time.
E. Nijkamp, B. Pang, H. Hayashi, L. Tu, H. Wang, Y. Zhou, S. Savarese, and C. Xiong, ‘‘Codegen: An open large language model for code with multi-turn program synthesis,’’ in The Eleventh International Conference on Learning Representations , 2023. [Online]. Available: https://openreview.net/forum?id=iaYcJKpY2B_
2023
Closest in time.