LLM Program Optimization via Retrieval Augmented Search

Understand

Recent work has demonstrated the potential of large language models (LLMs) for program optimization, a key challenge in programming languages.

• We propose a blackbox adaptation method called Retrieval Augmented Search (RAS) that performs beam search over candidate optimizations; at each step, it retrieves in-context examples from a given training dataset of slow-fast program pairs to guide the LLM.
• Critically, we find that performing contextual retrieval based on an LLM-generated natural language description significantly outperforms retrieval based on the source code.
• We also propose AEGIS, a method for improving interpretability by decomposing training examples into ''atomic edits'' that are significantly more incremental in nature.