Streaming 4D Visual Geometry Transformer

Understand

Perceiving and reconstructing 3D geometry from videos is a fundamental yet challenging computer vision task.

• To facilitate interactive and low-latency applications, we propose a streaming visual geometry transformer that shares a similar philosophy with autoregressive large language models.
• We explore a simple and efficient design and employ a causal transformer architecture to process the input sequence in an online manner.
• We use temporal causal attention and cache the historical keys and values as implicit memory to enable efficient streaming long-term 3D reconstruction.