Skill over Scale: The Case for Medium, Domain-Specific Models for SE

J. R. Landis and G. G. Koch, “An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers,” Biometrics , pp. 363–374, 1977

1977

Earlier work this paper cites.

(2013) Predict Closed Questions on Stack Overflow. [Online]. Available: https://www.kaggle.com/competitions/predict-closed-questions-on-stack-overflow

2013

Earlier work this paper cites.

S. Wang, D. Lo, and L. Jiang, “An empirical study on developer interactions in stackoverflow,” in Proceedings of the 28th annual ACM symposium on applied computing , 2013, pp. 1019–1024

2013

Earlier work this paper cites.

A. Barua, S. W. Thomas, and A. E. Hassan, “What are developers talking about? an analysis of topics and trends in stack overflow,” Empirical Software Engineering , vol. 19, pp. 619–654, 2014

2014

Earlier work this paper cites.

F. Fischer, K. Böttinger, H. Xiao, C. Stransky, Y. Acar, M. Backes, and S. Fahl, “Stack overflow considered harmful? the impact of copy&paste on android application security,” in 2017 IEEE Symposium on Security and Privacy (SP) . IEEE, 2017, pp. 121–136

2017

Earlier work this paper cites.

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in neural information processing systems , 2017, pp. 5998–6008

2017

Earlier work this paper cites.

I. K. Villanes, S. M. Ascate, J. Gomes, and A. C. Dias-Neto, “What are software engineers asking about android testing on stack overflow?” in Proceedings of the XXXI Brazilian Symposium on Software Engineering , 2017, pp. 104–113

2017

Earlier work this paper cites.

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805 , 2018

Original

2018

Earlier work this paper cites.

N. Meng, S. Nagy, D. Yao, W. Zhuang, and G. A. Argoty, “Secure coding practices in java: Challenges and vulnerabilities,” in Proceedings of the 40th International Conference on Software Engineering , 2018, pp. 372–383

2018

Earlier work this paper cites.

T. Kudo and J. Richardson, “Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing,” arXiv preprint arXiv:1808.06226 , 2018

Original

2018

Earlier work this paper cites.

M. Shoeybi, M. Patwary, R. Puri, P. LeGresley, J. Casper, and B. Catanzaro, “Megatron-lm: Training multi-billion parameter language models using model parallelism,” arXiv preprint arXiv:1909.08053 , 2019

Original

2019

Earlier work this paper cites.

C. Ragkhitwetsagul, J. Krinke, M. Paixao, G. Bianco, and R. Oliveto, “Toxic code snippets on stack overflow,” IEEE Transactions on Software Engineering , 2019

2019

Earlier work this paper cites.

C. Sun, X. Qiu, Y. Xu, and X. Huang, “How to fine-tune bert for text classification?” in Chinese Computational Linguistics: 18th China National Conference, CCL 2019, Kunming, China, October 18–20, 2019, Proceedings 18 . Springer, 2019, pp. 194–206

2019

Earlier work this paper cites.

K. Labusch, P. Kulturbesitz, C. Neudecker, and D. Zellhöfer, “Bert for named entity recognition in contemporary and historical german,” in Proceedings of the 15th conference on natural language processing, Erlangen, Germany , 2019, pp. 8–11

2019

Earlier work this paper cites.

H. Zhang, S. Wang, T.-H. P. Chen, Y. Zou, and A. E. Hassan, “An empirical study of obsolete answers on stack overflow,” IEEE Transactions on Software Engineering , 2019

2019

Earlier work this paper cites.

S. Gururangan, A. Marasović, S. Swayamdipta, K. Lo, I. Beltagy, D. Downey, and N. A. Smith, “Don’t stop pretraining: Adapt language models to domains and tasks,” arXiv preprint arXiv:2004.10964 , 2020

Original

2020

Earlier work this paper cites.

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al. , “Language models are few-shot learners,” Advances in neural information processing systems , vol. 33, pp. 1877–1901, 2020

2020

Earlier work this paper cites.

F. Liu, G. Li, Y. Zhao, and Z. Jin, “Multi-task learning based pre-trained language model for code completion,” in Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering , 2020, pp. 473–485

2020

Earlier work this paper cites.

R. Wang, H. Zhang, G. Lu, L. Lyu, and C. Lyu, “Fret: Functional reinforced transformer with bert for code summarization,” IEEE Access , vol. 8, pp. 135 591–135 604, 2020

2020

Cited alongside, same era.

Z. Feng, D. Guo, D. Tang, N. Duan, X. Feng, M. Gong, L. Shou, B. Qin, T. Liu, D. Jiang et al. , “Codebert: A pre-trained model for programming and natural languages,” arXiv preprint arXiv:2002.08155 , 2020

Original

2020

Cited alongside, same era.

J. Tabassum, M. Maddela, W. Xu, and A. Ritter, “Code and named entity recognition in stackoverflow,” arXiv preprint arXiv:2005.01634 , 2020

Original

2020

Cited alongside, same era.

J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei, “Scaling laws for neural language models,” arXiv preprint arXiv:2001.08361 , 2020

Original

2020

Cited alongside, same era.

I. Annamoradnejad, J. Habibi, and M. Fazli, “Multi-view approach to suggest moderation actions in community question answering sites,” Information Sciences , vol. 600, pp. 144–154, 2022. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0020025522003127

2022

Later among the works it cites.

S. Liu and A. Ritter, “Do conll-2003 named entity taggers still work well in 2023?” arXiv preprint arXiv:2212.09747 , 2022

Original

2022

Later among the works it cites.

D. Bleyl and E. K. Buxton, “Emotion recognition on stackoverflow posts using bert,” in 2022 IEEE International Conference on Big Data (Big Data) . IEEE, 2022, pp. 5881–5885

2022

Later among the works it cites.

J. He, B. Xu, Z. Yang, D. Han, C. Yang, and D. Lo, “Ptm4tag: sharpening tag recommendation of stack overflow posts with pre-trained models,” in Proceedings of the 30th IEEE/ACM International Conference on Program Comprehension , 2022, pp. 1–11

2022

Later among the works it cites.