Agent-SAMA: State-Aware Mobile Assistant

Understand

Mobile Graphical User Interface (GUI) agents aim to autonomously complete tasks within or across apps based on user instructions.

• While recent Multimodal Large Language Models (MLLMs) enable these agents to interpret UI screens and perform actions, existing agents remain fundamentally reactive.
• They reason over the current UI screen but lack a structured representation of the app navigation flow, limiting GUI agents' ability to understand execution context, detect unexpected execution results, and recover from errors.
• We introduce Agent-SAMA, a state-aware multi-agent framework that models app execution as a Finite State Machine (FSM), treating UI screens as states and user actions as transitions.