Optimizing Large Language Model Hyperparameters for Code Generation

D. Binkley, M. Davis, D. Lawrie, and C. Morrell, “To camelcase or under_score,” in 2009 IEEE 17th International Conference on Program Comprehension , 2009, pp. 158–167

2009

Earlier work this paper cites.

I. Cohen, Y. Huang, J. Chen, J. Benesty, J. Benesty, J. Chen, Y. Huang, and I. Cohen, “Pearson correlation coefficient,” Noise reduction in speech processing , pp. 1–4, 2009

2009

Earlier work this paper cites.

L. Breiman, Classification and regression trees . Routledge, 2017

2017

Earlier work this paper cites.

M. Feurer and F. Hutter, “Hyperparameter optimization,” Automated machine learning: Methods, systems, challenges , pp. 3–33, 2019

2019

Earlier work this paper cites.

M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Herbert-Voss, W. H. Guss, A. Nichol, A. Paino, N. Tezak, J. Tang, I. Babuschkin, S. Balaji, S. Jain, W. Saunders, C. Hesse, A. N. Carr, J. Leike, J. Achiam, V. Misra, E. Morikawa, A. Radford, M. Knight, M. Brundage, M. Murati, K. Mayer, P. Welinder, B. McGrew, D. Amodei, S. McCandlish, I. Sutskever, and W. Zaremba, “Evaluating large language models trained on code,” 2021

2021

Earlier work this paper cites.

N. Alexandr, O. Irina, K. Tatyana, K. Inessa, and P. Arina, “Fine-tuning gpt-3 for russian text summarization,” in Data Science and Intelligent Systems: Proceedings of 5th Computational Methods in Systems and Software 2021, Vol. 2 . Springer, 2021, pp. 748–757

2021

Earlier work this paper cites.

B. Bhavya, J. Xiong, and C. Zhai, “Analogy generation by prompting large language models: A case study of instructgpt,” 2022

2022

Earlier work this paper cites.

G. Poesia, A. Polozov, V. Le, A. Tiwari, G. Soares, C. Meek, and S. Gulwani, “Synchromesh: Reliable code generation from pre-trained language models,” in International Conference on Learning Representations , 2022. [Online]. Available: https://openreview.net/forum?id=KmtVD97J43e

2022

Earlier work this paper cites.

P. Bareiß, B. Souza, M. d’Amorim, and M. Pradel, “Code generation tools (almost) for free? a study of few-shot, pre-trained language models on code,” arXiv preprint arXiv:2206.01335 , 2022

Original

2022

Earlier work this paper cites.

X. Hou, Y. Zhao, Y. Liu, Z. Yang, K. Wang, L. Li, X. Luo, D. Lo, J. Grundy, and H. Wang, “Large language models for software engineering: A systematic literature review,” arXiv preprint arXiv:2308.10620 , 2023

Original

2023

Earlier work this paper cites.

A. Nguyen-Duc, B. Cabrero-Daniel, A. Przybylek, C. Arora, D. Khanna, T. Herda, U. Rafiq, J. Melegati, E. Guerra, K.-K. Kemell et al. , “Generative artificial intelligence for software engineering–a research agenda,” arXiv preprint arXiv:2310.18648 , 2023

Original

2023

Earlier work this paper cites.

X. Jiang, Y. Dong, L. Wang, Z. Fang, Q. Shang, G. Li, Z. Jin, and W. Jiao, “Self-planning code generation with large language models,” 2023

2023

Earlier work this paper cites.

X.-Y. Li, J.-T. Xue, Z. Xie, and M. Li, “Think outside the code: Brainstorming boosts large language models in code generation,” 2023

2023

Earlier work this paper cites.

C. Wang, S. X. Liu, and A. H. Awadallah, “Cost-effective hyperparameter optimization for large language model generation inference,” 2023. [Online]. Available: https://arxiv.org/abs/2303.04673

Original

2023

Earlier work this paper cites.

A. Buscemi, “A comparative study of code generation using chatgpt 3.5 across 10 programming languages,” 2023

2023

Cited alongside, same era.

Y. Feng, S. Vanam, M. Cherukupally, W. Zheng, M. Qiu, and H. Chen, “Investigating code generation performance of chatgpt with crowdsourcing social data,” in 2023 IEEE 47th Annual Computers, Software, and Applications Conference (COMPSAC) , 2023, pp. 876–885

2023

Cited alongside, same era.

I. G. Ndukwe, S. A. Licorish, A. Tahir, and S. G. MacDonell, “How have views on software quality differed over time? research and practice viewpoints,” Journal of Systems and Software , vol. 195, p. 111524, 2023. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S016412122200200X

2023

Cited alongside, same era.

S. Ouyang, J. M. Zhang, M. Harman, and M. Wang, “Llm is like a box of chocolates: the non-determinism of chatgpt in code generation,” 2023

2023

Cited alongside, same era.

X. Du, M. Liu, J. Li, H. Wang, X. Peng, and Y. Lou, “Resolving crash bugs via large language models: An empirical study,” 2023

2023

Later among the works it cites.

X. Chen, M. Lin, N. Schärli, and D. Zhou, “Teaching large language models to self-debug,” 2023

2023

Later among the works it cites.

J. Pan, A. Sadé, J. Kim, E. Soriano, G. Sole, and S. Flamant, “Stelocoder: a decoder-only llm for multi-language to python code translation,” 2023

2023

Later among the works it cites.

M. R. Zhang, N. Desai, J. Bae, J. Lorraine, and J. Ba, “Using large language models for hyperparameter optimization,” 2023

Original

2023

Later among the works it cites.

C. Arora, J. Grundy, and M. Abdelrazek, “Advancing requirements engineering through generative ai: Assessing the role of llms,” in Generative AI for Effective Software Development . Springer, 2024, pp. 129–148