The Larger the Better? Improved LLM Code-Generation via Budget Reallocation

Understand

It is a common belief that large language models (LLMs) are better than smaller-sized ones.

• However, larger models also require significantly more time and compute during inference.
• This begs the question: what happens when both models operate under the same budget? (e.g., compute, run-time).
• To address this question, we analyze code generation LLMs of various sizes and make comparisons such as running a 70B model once vs.