PointMapPolicy: Structured Point Cloud Processing for Multi-Modal Imitation Learning

Understand

Robotic manipulation systems benefit from complementary sensing modalities, where each provides unique environmental information.

• Point clouds capture detailed geometric structure, while RGB images provide rich semantic context.
• Current point cloud methods struggle to capture fine-grained detail, especially for complex tasks, which RGB methods lack geometric awareness, which hinders their precision and generalization.
• We introduce PointMapPolicy, a novel approach that conditions diffusion policies on structured grids of points without downsampling.