Fetching the paper…
Reading the bibliography…
Transformers have gained popularity in the software engineering (SE) literature.
Q. McNemar, “Note on the sampling error of the difference between correlated proportions or percentages,” Psychometrika , vol. 12, no. 2, pp. 153–157, 1947
1947
Earlier work this paper cites.
V. Levenshtein, “Binary Codes Capable of Correcting Deletions, Insertions and Reversals,” Soviet Physics Doklady , vol. 10, p. 707, 1966
1966
Earlier work this paper cites.
S. Holm, “A simple sequentially rejective multiple test procedure,” Scandinavian journal of statistics , pp. 65–70, 1979
1979
Earlier work this paper cites.
A. Robins, “Catastrophic forgetting, rehearsal and pseudorehearsal,” Connection Science , vol. 7, no. 2, pp. 123–146, 1995
1995
Earlier work this paper cites.
K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: A method for automatic evaluation of machine translation,” in Proceedings of the 40th Annual Meeting on Association for Computational Linguistics , ser. ACL ’02, 2002, pp. 311–318
2002
Earlier work this paper cites.
B. Kitchenham and S. Charters, “Guidelines for performing systematic literature reviews in software engineering,” 2007
2007
Earlier work this paper cites.
2009
Earlier work this paper cites.
M. Allamanis, H. Peng, and C. Sutton, “A convolutional attention network for extreme summarization of source code,” in International Conference on Machine Learning (ICML) , 2016
2016
Earlier work this paper cites.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in neural information processing systems , 2017, pp. 5998–6008
2017
Earlier work this paper cites.
G. Halevi, H. Moed, and J. Bar-Ilan, “Suitability of google scholar as a source of scientific information and as a source of data for scientific evaluation: Review of the literature,” Journal of Informetrics , vol. 11, no. 3, pp. 823–834, 2017. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S1751157717300676
2017
Earlier work this paper cites.
M. Tufano, F. Palomba, G. Bavota, M. D. Penta, R. Oliveto, A. D. Lucia, and D. Poshyvanyk, “There and back again: Can you compile that snapshot?” J. Softw. Evol. Process. , vol. 29, no. 4, 2017
2017
Earlier work this paper cites.
R. Robbes and A. Janes, “Leveraging small software engineering data sets with pre-trained neural networks,” in 2019 IEEE/ACM 41st International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) , 2019, pp. 29–32
2019
Earlier work this paper cites.
M. Tufano, C. Watson, G. Bavota, M. D. Penta, M. White, and D. Poshyvanyk, “An empirical study on learning bug-fixing patches in the wild via neural machine translation,” ACM Trans. Softw. Eng. Methodol. , vol. 28, no. 4, pp. 19:1–19:29, 2019
2019
Earlier work this paper cites.
S. Zafar, M. Z. Malik, and G. S. Walia, “Towards standardizing and improving classification of bug-fix commits,” in 2019 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM) . IEEE, 2019, pp. 1–6
2019
Earlier work this paper cites.
B. Wei, G. Li, X. Xia, Z. Fu, and Z. Jin, “Code generation as a dual task of code summarization,” Advances in neural information processing systems , vol. 32, 2019
2019
Earlier work this paper cites.
J. Devlin, M. Chang, K. Lee, and K. Toutanova, “BERT: pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT , J. Burstein, C. Doran, and T. Solorio, Eds. Association for Computational Linguistics, 2019, pp. 4171–4186
2019
Earlier work this paper cites.
A. LeClair and C. McMillan, “Recommendations for datasets for source code summarization,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , Jun. 2019, pp. 3931–3937
2019
Earlier work this paper cites.
2019
Earlier work this paper cites.
2019
Earlier work this paper cites.
A. Svyatkovskiy, S. K. Deng, S. Fu, and N. Sundaresan, “Intellicode compose: Code generation using transformer,” ser. ESEC/FSE 2020, 2020, pp. 1433–1443
2020
Earlier work this paper cites.
J. Zhang, Y. Zhao, M. Saleh, and P. J. Liu, “PEGASUS: pre-training with extracted gap-sentences for abstractive summarization,” in Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event , ser. Proceedings of Machine Learning Research, vol. 119. PMLR, 2020, pp. 11 328–11 339
2020
Earlier work this paper cites.
C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” Journal of Machine Learning Research , vol. 21, no. 140, pp. 1–67, 2020. [Online]. Available: http://jmlr.org/papers/v21/20-074.html
2020
Earlier work this paper cites.
P. He, C. Meister, and Z. Su, “Structure-invariant testing for machine translation,” in 2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE) . IEEE, 2020, pp. 961–973
2020
Earlier work this paper cites.
2020
Earlier work this paper cites.
F. Liu, G. Li, Y. Zhao, and Z. Jin, “Multi-task learning based pre-trained language model for code completion,” in Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering , 2020, pp. 473–485
2020
Earlier work this paper cites.
M. H. Asyrofi, F. Thung, D. Lo, and L. Jiang, “Crossasr: Efficient differential testing of automatic speech recognition via text-to-speech,” in 2020 IEEE International Conference on Software Maintenance and Evolution (ICSME) . IEEE, 2020, pp. 640–650
2020
Cited alongside, same era.
Z. Feng, D. Guo, D. Tang, N. Duan, X. Feng, M. Gong, L. Shou, B. Qin, T. Liu, D. Jiang, and M. Zhou, “CodeBERT: A pre-trained model for programming and natural languages,” in Findings of the Association for Computational Linguistics: EMNLP 2020 . Online: Association for Computational Linguistics, Nov. 2020, pp. 1536–1547. [Online]. Available: https://aclanthology.org/2020.findings-emnlp.139
2020
Cited alongside, same era.
T. H. Le, H. Chen, and M. A. Babar, “Deep learning for source code modeling and generation: Models, applications, and challenges,” ACM Computing Surveys (CSUR) , vol. 53, no. 3, pp. 1–38, 2020
2020
Cited alongside, same era.
D. Drain, C. Wu, A. Svyatkovskiy, and N. Sundaresan, “Generating bug-fixes using pretrained transformers,” ser. MAPS 2021, 2021, p. 1?8
X. Zhou, D. Han, and D. Lo, “Assessing generalizability of codebert,” in 2021 IEEE International Conference on Software Maintenance and Evolution (ICSME) , 2021, pp. 425–436
2021
Later among the works it cites.
2021
Later among the works it cites.
J. Henkel, D. Silva, L. Teixeira, M. d’Amorim, and T. Reps, “Shipwright: A human-in-the-loop system for dockerfile repair,” in 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) . IEEE, 2021, pp. 1148–1160
2021
Later among the works it cites.
G. Heyman, R. Huysegems, P. Justen, and T. Van Cutsem, “Natural language-guided programming,” in Proceedings of the 2021 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software , 2021, pp. 39–55
2021
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
2021
Cited alongside, same era.
B. Berabi, J. He, V. Raychev, and M. Vechev, “Tfix: Learning to fix coding errors with a text-to-text transformer,” in Proceedings of the 38th International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, M. Meila and T. Zhang, Eds., vol. 139, 2021, pp. 780–791
2021
Cited alongside, same era.
A. Mastropaolo, S. Scalabrino, N. Cooper, D. Nader Palacio, D. Poshyvanyk, R. Oliveto, and G. Bavota, “Studying the usage of text-to-text transfer transformer to support code-related tasks,” in 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) , 2021, pp. 336–347
2021
Cited alongside, same era.
E. Mashhadi and H. Hemmati, “Applying codebert for automated program repair of java simple bugs,” in 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR) , 2021, pp. 505–509
2021
Cited alongside, same era.
Z. Yang, J. Keung, X. Yu, X. Gu, Z. Wei, X. Ma, and M. Zhang, “A multi-modal transformer-based code summarization approach for smart contracts,” in 2021 IEEE/ACM 29th International Conference on Program Comprehension (ICPC) , 2021, pp. 1–12
2021
Cited alongside, same era.
H. Peng, G. Li, W. Wang, Y. Zhao, and Z. Jin, “Integrating tree path in transformer for code representation,” in Advances in Neural Information Processing Systems , M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, Eds., vol. 34. Curran Associates, Inc., 2021, pp. 9343–9354
2021
Cited alongside, same era.
Z. Tang, C. Li, J. Ge, X. Shen, Z. Zhu, and B. Luo, “Ast-transformer: Encoding abstract syntax trees efficiently for code summarization,” in 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE) , 2021, pp. 1193–1195
2021
Cited alongside, same era.
M. Ciniselli, N. Cooper, L. Pascarella, A. Mastropaolo, E. Aghajani, D. Poshyvanyk, M. Di Penta, and G. Bavota, “An empirical study on the usage of transformer models for code completion,” IEEE Transactions on Software Engineering , p. To appear, 2021
2021
Cited alongside, same era.
Z. Chen, S. Kommrusch, M. Tufano, L. Pouchet, D. Poshyvanyk, and M. Monperrus, “Sequencer: Sequence-to-sequence learning for end-to-end program repair,” IEEE Transactions on Software Engineering , vol. 47, no. 09, pp. 1943–1959, 2021
2021
Cited alongside, same era.
N. Jiang, T. Lutellier, and L. Tan, “Cure: Code-aware neural machine translation for automatic program repair,” in 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) . IEEE, 2021, pp. 1161–1173
2021
Later among the works it cites.
K. Pei, J. Guan, M. Broughton, Z. Chen, S. Yao, D. Williams-King, V. Ummadisetty, J. Yang, B. Ray, and S. Jana, “Stateformer: fine-grained type recovery from binaries using generative state modeling,” in Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering , 2021, pp. 690–702
2021
Later among the works it cites.
I. Ganguli, R. S. Bhowmick, S. Biswas, and J. Sil, “Empirical auto-evaluation of python code for performance analysis of transformer network using t5 architecture,” in 2021 8th International Conference on Smart Computing and Communications (ICSCC) , 2021, pp. 75–79
2021
Later among the works it cites.
B. Berabi, J. He, V. Raychev, and M. Vechev, “Tfix: Learning to fix coding errors with a text-to-text transformer,” in Proceedings of the 38th International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, M. Meila and T. Zhang, Eds., vol. 139. PMLR, 18–24 Jul 2021, pp. 780–791
2021
Later among the works it cites.
C. Pan, M. Lu, and B. Xu, “An empirical study on software defect prediction using codebert model,” Applied Sciences , vol. 11, no. 11, 2021
2021
Later among the works it cites.
O. Dabic, E. Aghajani, and G. Bavota, “Sampling projects in github for MSR studies,” in 18th IEEE/ACM International Conference on Mining Software Repositories, MSR 2021, Madrid, Spain, May 17-19, 2021 . IEEE, 2021, pp. 560–564
2021
Later among the works it cites.
F. Ferreira, L. L. Silva, and M. T. Valente, “Software engineering meets deep learning: a mapping study,” in Proceedings of the 36th Annual ACM Symposium on Applied Computing , 2021, pp. 1542–1549
2021
Later among the works it cites.
M. Paltenghi and M. Pradel, “Thinking like a developer? comparing the attention of humans with neural models of code,” in 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE) , 2021, pp. 867–879
2021
Later among the works it cites.
2021
Later among the works it cites.
R. Tufano, S. Masiero, A. Mastropaolo, L. Pascarella, D. Poshyvanyk, and G. Bavota, “Using pre-trained models to boost code review automation,” in 44th IEEE/ACM 44th International Conference on Software Engineering, ICSE 2022, Pittsburgh, PA, USA, May 25-27, 2022 . ACM, 2022, pp. 2291–2302
2022
Later among the works it cites.
P. Thongtanunam, C. Pornprasit, and C. Tantithamthavorn, “Autotransform: Automated code transformation to support modern code review process,” in 2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE) , 2022, pp. 237–248
2022
Later among the works it cites.
L. Zhiyu, L. Shuai, G. Daya, D. Nan, J. Shailesh, J. Grant, M. Deep, G. Jared, S. Alexey, F. Shengyu, and N. Sundaresan, “Automating code review activities by large-scale pre-training,” in 2022 ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) , 2022
2022
Later among the works it cites.
A. Mastropaolo, N. Cooper, D. Nader-Palacio, S. Scalabrino, D. Poshyvanyk, R. Oliveto, and G. Bavota, “Using transfer learning for code-related tasks,” IEEE Transactions on Software Engineering , 2022
2022
Later among the works it cites.
“Replication Package,” https://github.com/RosaliaTufano/impact_pre-training , 2022
2022
Later among the works it cites.
W. Alhoshan, L. Zhao, A. Ferrari, and K. J. Letsholo, “A zero-shot learning approach to classifying requirements: A preliminary study,” in Requirements Engineering: Foundation for Software Quality: 28th International Working Conference, REFSQ 2022, Proceedings , 2022, p. 52?59
2022
Later among the works it cites.
F. Zhang, X. Yu, J. Keung, F. Li, Z. Xie, Z. Yang, C. Ma, and Z. Zhang, “Improving stack overflow question title generation with copying enhanced codebert model and bi-modal information,” Information and Software Technology , vol. 148, p. 106922, 2022
2022
Later among the works it cites.
“langid.py,” https://github.com/saffsd/langid.py , accessed:2022-05-22
2022
Later among the works it cites.
“pycld3,” https://pypi.org/project/pycld3/ , accessed: 2022-05-22
2022
Later among the works it cites.
C. Watson, N. Cooper, D. N. Palacio, K. Moran, and D. Poshyvanyk, “A systematic literature review on the use of deep learning in software engineering research,” ACM Transactions on Software Engineering and Methodology (TOSEM) , vol. 31, no. 2, pp. 1–58, 2022
2022
Later among the works it cites.
F. Chen, F. Fard, D. Lo, and T. Bryksin, “On the transferability of pre-trained language models for low-resource programming languages,” in 2022 IEEE/ACM International Conference on Program Comprehension (ICPC) , 2022, p. To appear
2022
Later among the works it cites.