GAF: Gaussian Action Field as a 4D Representation for Dynamic World Modeling in Robotic Manipulation

Understand

Accurate scene perception is critical for vision-based robotic manipulation.

• Existing approaches typically follow either a Vision-to-Action (V-A) paradigm, predicting actions directly from visual inputs, or a Vision-to-3D-to-Action (V-3D-A) paradigm, leveraging intermediate 3D representations.
• However, these methods often struggle with action inaccuracies due to the complexity and dynamic nature of manipulation scenes.
• In this paper, we adopt a V-4D-A framework that enables direct action reasoning from motion-aware 4D representations via a Gaussian Action Field (GAF).