Software Engineering and Foundation Models: Insights from Industry Blogs Using a Jury of Foundation Models

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al. , “Language Models are Few-Shot Learners,” in Advances in Neural Information Processing Systems (NeurIPS) , vol. 33. Curran Associates, Inc., 2020, pp. 1877–1901

1901

Earlier work this paper cites.

K. E. Emam, “Benchmarking Kappa: Interrater Agreement in Software Process Assessments,” Empirical Software Engineering , vol. 4, no. 2, pp. 113–133, Jun 1999

1999

Earlier work this paper cites.

G. Butler, Think write grow: how to become a thought leader and build your business by creating exceptional articles, blogs, speeches, books and more . John Wiley & Sons, 2012

2012

Earlier work this paper cites.

Z. M. Jiang and A. E. Hassan, “A Survey on Load Testing of Large-Scale Software Systems,” IEEE Transactions on Software Engineering , vol. 41, no. 11, pp. 1091–1118, 2015

2015

Earlier work this paper cites.

S. Masuda, K. Ono, T. Yasue, and N. Hosokawa, “A Survey of Software Quality for Machine Learning Applications,” in IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) , 2018, pp. 279–284

2018

Earlier work this paper cites.

S. Amershi, A. Begel, C. Bird, R. DeLine, H. Gall, E. Kamar, N. Nagappan, B. Nushi, and T. Zimmermann, “Software Engineering for Machine Learning: A Case Study,” in IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) , 2019, pp. 291–300

2019

Earlier work this paper cites.

A. A. Bangash, H. Sahar, S. Chowdhury, A. W. Wong, A. Hindle, and K. Ali, “What do Developers Know About Machine Learning: A Study of ML Discussions on StackOverflow,” in IEEE/ACM 16th International Conference on Mining Software Repositories (MSR) , 2019, pp. 260–264

2019

Earlier work this paper cites.

P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel, S. Riedel, and D. Kiela, “Retrieval-augmented generation for knowledge-intensive NLP tasks,” in Proceedings of the 34th International Conference on Neural Information Processing Systems (NeurIPS) . Curran Associates Inc., 2020

2020

Earlier work this paper cites.

R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-Resolution Image Synthesis with Latent Diffusion Models,” arXiv preprint arXiv:2112.10752 , 2021

Original

2021

Earlier work this paper cites.

H. Villamizar, T. Escovedo, and M. Kalinowski, “Requirements Engineering for Machine Learning: A Systematic Mapping Study,” in 47th Euromicro Conference on Software Engineering and Advanced Applications (SEAA) , 2021, pp. 29–36

2021

Earlier work this paper cites.

H. Chase, “LangChain,” https://github.com/langchain-ai/langchain , last visited: Oct 11, Oct. 2022

2022

Earlier work this paper cites.

O. Khattab, K. Santhanam, X. L. Li, D. Hall, P. Liang, C. Potts, and M. Zaharia, “Demonstrate-search-predict: Composing retrieval and language models for knowledge-intensive NLP,” arXiv preprint arXiv:2212.14024 , 2022

Original

2022

Earlier work this paper cites.

S. Lin, J. Hilton, and O. Evans, “Teaching Models to Express Their Uncertainty in Words,” Transactions on Machine Learning Research , 2022

2022

Earlier work this paper cites.

S. Mangrulkar, S. Gugger, L. Debut, Y. Belkada, S. Paul, and B. Bossan, “Peft: State-of-the-art parameter-efficient fine-tuning methods,” https://github.com/huggingface/peft , last visited: Oct 11, 2022

2022

Earlier work this paper cites.

S. Martínez-Fernández, J. Bogner, X. Franch, M. Oriol, J. Siebert, A. Trendowicz, A. M. Vollmer, and S. Wagner, “Software Engineering for AI-Based Systems: A Survey,” ACM Trans. Softw. Eng. Methodol. , vol. 31, no. 2, apr 2022

2022

Earlier work this paper cites.

S. J. Mielke, A. Szlam, E. Dinan, and Y.-L. Boureau, “Reducing Conversational Agents’ Overconfidence Through Linguistic Calibration,” Transactions of the Association for Computational Linguistics , vol. 10, pp. 857–872, 2022

2022

Earlier work this paper cites.

L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al. , “Training language models to follow instructions with human feedback,” in Advances in Neural Information Processing Systems (NeurIPS) , vol. 35. Curran Associates, Inc., 2022, pp. 27 730–27 744

2022

Earlier work this paper cites.

J. Wei, Y. Tay, R. Bommasani, C. Raffel, B. Zoph, S. Borgeaud, D. Yogatama, M. Bosma, D. Zhou, D. Metzler et al. , “Emergent Abilities of Large Language Models,” arXiv preprint arXiv:2206.07682 , 2022

Original

2022

Cited alongside, same era.

J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. H. Chi, Q. V. Le, and D. Zhou, “Chain-of-thought prompting elicits reasoning in large language models,” in Proceedings of the 36th International Conference on Neural Information Processing Systems (NeurIPS) . Curran Associates Inc., 2022

2022

Cited alongside, same era.

C. Chen, S. Borgeaud, G. Irving, J.-B. Lespiau, L. Sifre, and J. Jumper, “Accelerating Large Language Model Decoding with Speculative Sampling,” arXiv preprint arXiv:2302.01318 , 2023

Original

2023

Cited alongside, same era.

A. Fan, B. Gokkaya, M. Harman, M. Lyubarskiy, S. Sengupta, S. Yoo, and J. M. Zhang, “Large Language Models for Software Engineering: Survey and Open Problems,” in IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE) , May 2023

A. E. Hassan, D. Lin, G. K. Rajbahadur, K. Gallaba, F. R. Cogo, B. Chen, H. Zhang, K. Thangarajah, G. Oliva, J. J. Lin et al. , “Rethinking Software Engineering in the Era of Foundation Models: A Curated Catalogue of Challenges in the Development of Trustworthy FMware,” in Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering (FSE) , 2024, p. 294–305

2024

Closest in time.

A. E. Hassan, G. A. Oliva, D. Lin, B. Chen, Z. Ming, and Jiang, “Towards AI-Native software engineering (SE 3.0): A vision and a challenge roadmap,” arXiv preprint arXiv:2410.06107 , 2024

Original

2024

Closest in time.

X. Hou, Y. Zhao, Y. Liu, Z. Yang, K. Wang, L. Li, X. Luo, D. Lo, J. Grundy, and H. Wang, “Large Language Models for Software Engineering: A Systematic Literature Review,” ACM Trans. Softw. Eng. Methodol. , Sep. 2024

2024

Closest in time.

H. Li, C.-P. Bezemer, and A. E. Hassan, “Replication package,” https://github.com/SAILResearch/fmse-blogs , 2024