Fetching the paper…
Reading the bibliography…
The use of modern Natural Language Processing (NLP) techniques has shown to be beneficial for software engineering tasks, such as vulnerability detection and type inference.
Linguistic Knowledge and Transferability of Contextual Representations
Nelson F. Liu, Matt Gardner, Yonatan Belinkov, Matthew E. Peters, and Noah A. Smith. 2019a · 1903
Earlier work this paper cites.
How to Fine-Tune BERT for Text Classification?
Chi Sun, Xipeng Qiu, Yige Xu, and Xuanjing Huang. 2020 · 1905
Earlier work this paper cites.
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2020 · 1906
Earlier work this paper cites.
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019b · 1907
Earlier work this paper cites.
Reducing Transformer Depth on Demand with Structured Dropout
Angela Fan, Edouard Grave, and Armand Joulin. 2019 · 1909
Earlier work this paper cites.
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020 · 1909
Earlier work this paper cites.
Yaqin Zhou, Shangqing Liu, Jingkai Siow, Xiaoning Du, and Yang Liu. 2019 · 1909
Earlier work this paper cites.
Individual Comparisons by Ranking Methods
Frank Wilcoxon. 1992 · 1992
Earlier work this paper cites.
A Critique and Improvement of the CL Common Language Effect Size Statistics of McGraw and Wong
András Vargha and Harold D. Delaney. 2000 · 2000
Earlier work this paper cites.
Aditya Kanade, Petros Maniatis, Gogul Balakrishnan, and Kensen Shi. 2020 · 2001
Earlier work this paper cites.
Deep RNNs Encode Soft Hierarchical Syntax. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) . Association for Computational Linguistics, Melbourne, Australia, 14–19
Terra Blevins, Omer Levy, and Luke Zettlemoyer. 2018 · 2003
Earlier work this paper cites.
On the Effect of Dropping Layers of Pre-trained Transformer Models
Hassan Sajjad, Fahim Dalvi, Nadir Durrani, and Preslav Nakov. 2023 · 2004
Earlier work this paper cites.
Revisiting Few-sample BERT Fine-tuning. In NeurIPS 2021 . 1–22
Tianyi Zhang, Felix Wu, Arzoo Katiyar, Kilian Q. Weinberger, and Yoav Artzi. 2021 · 2006
Earlier work this paper cites.
Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, Michele Tufano, Shao Kun Deng, Colin Clement, Dawn Drain, Neel Sundaresan, Jian Yin, Daxin Jiang, and Ming Zhou. 2021 · 2009
Earlier work this paper cites.
New Initiative: The Naturalness of Software. In Proceedings - International Conference on Software Engineering , Vol. 2. IEEE Computer Society, 543–546
P. Devanbu. 2015 · 2015
Earlier work this paper cites.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Earlier work this paper cites.
A Survey of Machine Learning for Big Code and Naturalness
Miltiadis Allamanis, Earl T. Barr, Premkumar Devanbu, and Charles Sutton. 2018 · 2018
Earlier work this paper cites.
Deep Learning Type Inference. In Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) . ACM, New York, NY, USA, 152–162
Vincent J. Hellendoorn, Christian Bird, Earl T. Barr, and Miltiadis Allamanis. 2018 · 2018
Cited alongside, same era.
Universal Language Model Fine-tuning for Text Classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Association for Computational Linguistics, Melbourne, Australia, 328–339
Jeremy Howard and Sebastian Ruder. 2018 · 2018
Cited alongside, same era.
Dissecting Contextual Word Embeddings: Architecture and Representation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing . Association for Computational Linguistics, Brussels, Belgium, 1499–1509
Matthew Peters, Mark Neumann, Luke Zettlemoyer, and Wen-tau Yih. 2018 · 2018
Cited alongside, same era.
Automated Vulnerability Detection in Source Code Using Deep Representation Learning. In International Conference on Machine Learning and Applications (ICMLA) . IEEE, Orlando, FL, 757–762
Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin Clement, Dawn Drain, Daxin Jiang, Duyu Tang, Ge Li, Lidong Zhou, Linjun Shou, Long Zhou, Michele Tufano, Ming Gong, Ming Zhou, Nan Duan, Neel Sundaresan, Shao Kun Deng, Shengyu Fu, and Shujie Liu. 2021 · 2021
Later among the works it cites.
Thinking Like a Developer? Comparing the Attention of Humans with Neural Models of Code. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE) . 867–879
Matteo Paltenghi and Michael Pradel. 2021 · 2021
Later among the works it cites.
An Empirical Study on Software Defect Prediction Using CodeBERT Model
Cong Pan, Minyan Lu, and Biao Xu. 2021 · 2021
Later among the works it cites.
CoTexT: Multi-task Learning with Code-Text Transformer. In Workshop on Natural Language Processing for Programming (NLP4Prog 2021) . Association for Computational Linguistics, Online, 40–47
Long Phan, Hieu Tran, Daniel Le, Hieu Nguyen, James Annibal, Alec Peltekian, and Yanfang Ye. 2021 · 2021
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Rebecca Russell, Louis Kim, Lei Hamilton, Tomo Lazovich, Jacob Harer, Onur Ozdemir, Paul Ellingwood, and Marc McConley. 2018 · 2018
Cited alongside, same era.
DeepSim: Deep Learning Code Functional Similarity. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering . ACM, Lake Buena Vista FL USA, 141–151
Gang Zhao and Jeff Huang. 2018 · 2018
Cited alongside, same era.
SEQUENCER: Sequence-to-Sequence Learning for End-to-End Program Repair
Zimin Chen, Steve James Kommrusch, Michele Tufano, Louis-Noël Pouchet, Denys Poshyvanyk, and Martin Monperrus. 2019 · 2019
Cited alongside, same era.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL, Vol 1) . Association for Computational Linguistics, 4171–4186
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Cited alongside, same era.
To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks. In Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019) . Association for Computational Linguistics, Florence, Italy, 7–14
Matthew E. Peters, Sebastian Ruder, and Noah A. Smith. 2019 · 2019
Cited alongside, same era.
How to Fine-Tune BERT for Text Classification?. In Chinese Computational Linguistics (Lecture Notes in Computer Science) , Maosong Sun, Xuanjing Huang, Heng Ji, Zhiyuan Liu, and Yang Liu (Eds.). Springer International Publishing, Cham, 194–206
Chi Sun, Xipeng Qiu, Yige Xu, and Xuanjing Huang. 2019 · 2019
Cited alongside, same era.
PyMT5: Multi-Mode Translation of Natural Language and Python Code with Transformers. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) . Association for Computational Linguistics, Online, 9052–9065
Colin Clement, Dawn Drain, Jonathan Timcheck, Alexey Svyatkovskiy, and Neel Sundaresan. 2020 · 2020
Cited alongside, same era.
CodeBERT: A Pre-Trained Model for Programming and Natural Languages. In Findings of the Association for Computational Linguistics: EMNLP 2020 . Online, 1536–1547
Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. 2020 · 2020
Cited alongside, same era.
Unified Pre-training for Program Understanding and Generation. In Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies . Association for Computational Linguistics, Online, 2655–2668
Wasi Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai-Wei Chang. 2021 · 2021
Cited alongside, same era.
Later among the works it cites.
A Survey on Machine Learning Techniques for Source Code Analysis
Tushar Sharma, Maria Kechagia, Stefanos Georgiou, Rohit Tiwari, and Federica Sarro. 2021 · 2021
Later among the works it cites.
CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation. In Conference on Empirical Methods in Natural Language Processing . Association for Computational Linguistics, Online and Punta Cana, Dominican Republic, 8696–8708
Yue Wang, Weishi Wang, Shafiq Joty, and Steven C.H. Hoi. 2021 · 2021
Later among the works it cites.
A Context-Aware Neural Embedding for Function-Level Vulnerability Detection
Hongwei Wei, Guanjun Lin, Lin Li, and Heming Jia. 2021 · 2021
Later among the works it cites.
Michihiro Yasunaga and Percy Liang. 2021 · 2021
Later among the works it cites.
Multilingual Training for Software Engineering. In Proceedings of the 44th International Conference on Software Engineering . ACM, Pittsburgh Pennsylvania, 1443–1455
Toufique Ahmed and Premkumar Devanbu. 2022 · 2022
Later among the works it cites.
VulRepair: A T5-based Automated Software Vulnerability Repair. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2022) . Association for Computing Machinery, New York, NY, USA, 935–947
Michael Fu, Chakkrit Tantithamthavorn, Trung Le, Van Nguyen, and Dinh Phung. 2022 · 2022
Later among the works it cites.
Deep Learning Meets Software Engineering: A Survey on Pre-Trained Models of Source Code
Changan Niu, Chuanyi Li, Bin Luo, and Vincent Ng. 2022 · 2022
Later among the works it cites.
Greedy-Layer Pruning: Speeding up Transformer Models for Natural Language Processing
David Peer, Sebastian Stabinger, Stefan Engl, and Antonio Rodriguez-Sanchez. 2022 · 2022
Later among the works it cites.
An Exploratory Study on Code Attention in BERT. In Proceedings of the 30th IEEE/ACM International Conference on Program Comprehension (ICPC ’22) . Association for Computing Machinery, New York, NY, USA, 437–448
Rishab Sharma, Fuxiang Chen, Fatemeh Fard, and David Lo. 2022 · 2022
Later among the works it cites.
Neural Program Repair with Execution-Based Backpropagation. In Proceedings of the 44th International Conference on Software Engineering (ICSE ’22) . Association for Computing Machinery, New York, NY, USA, 1506–1518
He Ye, Matias Martinez, and Martin Monperrus. 2022 · 2022
Later among the works it cites.
AST-Probe: Recovering Abstract Syntax Trees from Hidden Representations of Pre-Trained Language Models. In Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering (ASE ’22) . Association for Computing Machinery, New York, NY, USA, 1–11
José Antonio Hernández López, Martin Weyssow, Jesús Sánchez Cuadrado, and Houari Sahraoui. 2023 · 2023
Closest in time.
StarCoder: May the Source Be with You!
Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, Qian Liu, Evgenii Zheltonozhskii, Terry Yue Zhuo, Thomas Wang, Olivier Dehaene, Mishig Davaadorj, Joel Lamy-Poirier, João Monteiro, Oleh Shliazhko, Nicolas Gontier, Nicholas Meade, Armel Zebaze, Ming-Ho Yee, Logesh Kumar Umapathi, Jian Zhu, Benjamin Lipkin, Muhtasham Oblokulov, Zhiruo Wang, Rudra Murthy, Jason Stillerman, Siva Sankalp Patel, Dmitry Abulkhanov, Marco Zocca, Manan Dey, Zhihan Zhang, Nour Fahmy, Urvashi Bhattacharyya, Wenhao Yu, Swayam Singh, Sasha Luccioni, Paulo Villegas, Maxim Kunakov, Fedor Zhdanov, Manuel Romero, Tony Lee, Nadav Timor, Jennifer Ding, Claire Schlesinger, Hailey Schoelkopf, Jan Ebert, Tri Dao, Mayank Mishra, Alex Gu, Jennifer Robinson, Carolyn Jane Anderson, Brendan Dolan-Gavitt, Danish Contractor, Siva Reddy, Daniel Fried, Dzmitry Bahdanau, Yacine Jernite, Carlos Muñoz Ferrandis, Sean Hughes, Thomas Wolf, Arjun Guha, Leandro von Werra, and Harm de Vries. 2023 · 2023
Closest in time.
ContraBERT: Enhancing Code Pre-Trained Models via Contrastive Learning. In Proceedings of the 45th International Conference on Software Engineering (ICSE ’23) . IEEE Press, Melbourne, Victoria, Australia, 2476–2487
Shangqing Liu, Bozhi Wu, Xiaofei Xie, Guozhu Meng, and Yang Liu. 2023 · 2023
Closest in time.