The Tyranny of Possibilities in the Design of Task-Oriented LLM Systems: A Scoping Survey

Understand

This scoping survey focuses on our current understanding of the design space for task-oriented LLM systems and elaborates on definitions and relationships among the available design parameters.

• The paper begins by defining a minimal task-oriented LLM system and exploring the design space of such systems through a thought experiment contemplating the performance of diverse LLM system configurations (involving single LLMs, single LLM-based agents, and multiple LLM-based agent systems) on a complex software development task and hypothesizes the results.
• We discuss a pattern in our results and formulate them into three conjectures.
• While these conjectures may be partly based on faulty assumptions, they provide a starting point for future research.