Prompt Adaptation as a Dynamic Complement in Generative AI Systems

Understand

As generative AI systems rapidly improve, a key question emerges: how do users adapt to these changes, and when does such adaptation matter for realizing performance gains? Drawing on theories of dynamic capabilities and IT complements, we study prompt adaptation--how users adjust their inputs in response to evolving model behavior--using a common experimental design applied to two preregistered tasks with 3,750 total participants who submitted nearly 37,000 prompts.

• We show that the importance of prompt adaptation depends critically on task structure.
• In a task with fixed evaluation criteria and an unambiguous goal, user prompt adaptation accounts for roughly half of the performance gains from a model upgrade.
• In contrast, in an open-ended creative task where the space of acceptable outputs is effectively unbounded and quality is subjective, performance improvements are driven primarily by model capability; prompt adaptation plays a limited role.