MobileSafetyBench: Evaluating Safety of Autonomous Agents in Mobile Device Control

Understand

Autonomous agents powered by large language models (LLMs) show promising potential in assistive tasks across various domains, including mobile device control.

• As these agents interact directly with personal information and device settings, ensuring their safe and reliable behavior is crucial to prevent undesirable outcomes.
• However, no benchmark exists for standardized evaluation of the safety of mobile device-control agents.
• In this work, we introduce MobileSafetyBench, a benchmark designed to evaluate the safety of device-control agents within a realistic mobile environment based on Android emulators.