EFM3D: A Benchmark for Measuring Progress Towards 3D Egocentric Foundation Models

Understand

The advent of wearable computers enables a new source of context for AI that is embedded in egocentric sensor data.

• This new egocentric data comes equipped with fine-grained 3D location information and thus presents the opportunity for a novel class of spatial foundation models that are rooted in 3D space.
• To measure progress on what we term Egocentric Foundation Models (EFMs) we establish EFM3D, a benchmark with two core 3D egocentric perception tasks.
• EFM3D is the first benchmark for 3D object detection and surface regression on high quality annotated egocentric data of Project Aria.