ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agents

Understand

Autonomous web agents solve complex browsing tasks, yet existing benchmarks measure only whether an agent finishes a task, ignoring whether it does so safely or in a way enterprises can trust.

• To integrate these agents into critical workflows, safety and trustworthiness (ST) are prerequisite conditions for adoption.
• We introduce \textbf{\textsc{ST-WebAgentBench}}, a configurable and easily extensible suite for evaluating web agent ST across realistic enterprise scenarios.
• Each of its 222 tasks is paired with ST policies, concise rules that encode constraints, and is scored along six orthogonal dimensions (e.g., user consent, robustness).