PerLA: Perceptive 3D Language Assistant

Understand

Enabling Large Language Models (LLMs) to understand the 3D physical world is an emerging yet challenging research direction.

• Current strategies for processing point clouds typically downsample the scene or divide it into smaller parts for separate analysis.
• However, both approaches risk losing key local details or global contextual information.
• In this paper, we introduce PerLA, a 3D language assistant designed to be more perceptive to both details and context, making visual representations more informative for the LLM.