CoAct-1: Computer-using Multi-Agent System with Coding Actions

Understand

Autonomous agents that operate computers via Graphical User Interfaces (GUIs) often struggle with efficiency and reliability on complex, long-horizon tasks.

• While augmenting these agents with planners can improve task decomposition, they remain constrained by the inherent limitations of performing all actions through GUI manipulation, leading to brittleness and inefficiency.
• In this work, we introduce a more robust and flexible paradigm: enabling agents to use coding as a enhanced action.
• We present CoAct-1, a novel multi-agent system that synergistically combines GUI-based control with direct programmatic execution.