Effective LLM-Driven Code Generation with Pythoness

G. Fink and M. Bishop, “Property-Based Testing: A New Approach to Testing for Assurance,” ACM SIGSOFT Softw. Eng. Notes , vol. 22, no. 4, pp. 74–80, 1997

1997

Earlier work this paper cites.

A. Solar-Lezama, R. M. Rabbah, R. Bodík, and K. Ebcioglu, “Programming by Sketching for Bit-Streaming Programs,” in Proceedings of the ACM SIGPLAN 2005 Conference on Programming Language Design and Implementation . ACM, 2005, pp. 281–294

2005

Earlier work this paper cites.

A. Solar-Lezama, “The Sketching Approach to Program Synthesis,” in Proceedings of the Asian Symposium on Programming Languages and Systems , ser. Lecture Notes in Computer Science, vol. 5904. Springer, 2009, pp. 4–13

2009

Earlier work this paper cites.

D. Maciver and Z. Hatfield-Dodds, “Hypothesis: A New Approach to Property-Based Testing,” J. Open Source Softw. , vol. 4, no. 43, p. 1891, 2019

2019

Earlier work this paper cites.

OpenAI, “Evaluating Large Language Models Trained on Code,” 2021

2021

Earlier work this paper cites.

P. Vaithilingam, T. Zhang, and E. L. Glassman, “Expectation vs. experience: Evaluating the usability of code generation tools powered by large language models,” in CHI ’22: CHI Conference on Human Factors in Computing Systems, Extended Abstracts , 2022, pp. 332:1–332:7

2022

Earlier work this paper cites.

E. Kalliamvakou, “Research: Quantifying GitHub Copilot’s Impact on Developer Productivity and Happiness,” The GitHub Blog , 2022

2022

Earlier work this paper cites.

S. Barke, M. B. James, and N. Polikarpova, “Grounded copilot: How programmers interact with code-generating models,” Proc. ACM Program. Lang. , vol. 7, no. OOPSLA1, pp. 85–111, 2023

2023

Cited alongside, same era.

B. Chopra, A. Singha, A. Fariha, S. Gulwani, C. Parnin, A. Tiwari, and A. Z. Henley, “Conversational Challenges in AI-Powered Data Science: Obstacles, Needs, and Design Opportunities,” arXiv preprint arXiv:2310.16164 , 2023

Original

2023

Cited alongside, same era.

E. Zelikman, Q. Huang, G. Poesia, N. D. Goodman, and N. Haber, “Parsel: Algorithmic reasoning with language models by composing decompositions,” in Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023 , A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, Eds., 2023. [Online]. Available: http://papers.nips.cc/paper_files/paper/2023/hash/6445dd88ebb9a6a3afa0b126ad87fe41-Abstract-Conference.html

2023

Cited alongside, same era.

E. Paradis, K. Grey, Q. Madison, D. Nam, A. Macvean, V. Meimand, N. Zhang, B. Ferrari-Church, and S. Chandra, “How much does AI impact development speed? An enterprise-based randomized controlled trial,” 2024. [Online]. Available: https://arxiv.org/abs/2410.12944

Original

2024

Later among the works it cites.

J. T. Liang, C. Yang, and B. A. Myers, “A Large-Scale Survey on the Usability of AI Programming Assistants: Successes and Challenges,” in Proceedings of the 46th IEEE/ACM International Conference on Software Engineering , 2024, pp. 1–13

2024

Later among the works it cites.

(2024) Pythia. Accessed: 2024-11-16. [Online]. Available: https://en.wikipedia.org/wiki/Pythia

2024

Later among the works it cites.

(2024) LeetCode. Accessed: 2024-11-16. [Online]. Available: https://leetcode.com/problemset/

2024

Later among the works it cites.

——. (2024) Hello GPT-4o. Accessed: 2024-11-16. [Online]. Available: https://openai.com/index/hello-gpt-4o/