Zero-Shot Visual Generalization in Robot Manipulation

Understand

Training vision-based manipulation policies that are robust across diverse visual environments remains an important and unresolved challenge in robot learning.

• Current approaches often sidestep the problem by relying on invariant representations such as point clouds and depth, or by brute-forcing generalization through visual domain randomization and/or large, visually diverse datasets.
• Disentangled representation learning - especially when combined with principles of associative memory - has recently shown promise in enabling vision-based reinforcement learning policies to be robust to visual distribution shifts.
• However, these techniques have largely been constrained to simpler benchmarks and toy environments.