Are My Optimized Prompts Compromised? Exploring Vulnerabilities of LLM-based Optimizers

Understand

Large language model (LLM) systems increasingly power everyday AI applications such as chatbots, computer-use assistants, and autonomous robots, where performance often depends on manually well-crafted prompts.

• LLM-based prompt optimizers reduce that effort by iteratively refining prompts from scored feedback, yet the security of this optimization stage remains underexamined.
• We present the first systematic analysis of poisoning risks in LLM-based prompt optimization.
• Using HarmBench, we find systems are substantially more vulnerable to manipulated feedback than to query poisoning alone: feedback-based attacks raise attack success rate (ASR) by up to {\Delta}ASR = 0.48.