PARIS3D: Reasoning-based 3D Part Segmentation Using Large Multimodal Model

Understand

Recent advancements in 3D perception systems have significantly improved their ability to perform visual recognition tasks such as segmentation.

• However, these systems still heavily rely on explicit human instruction to identify target objects or categories, lacking the capability to actively reason and comprehend implicit user intentions.
• We introduce a novel segmentation task known as reasoning part segmentation for 3D objects, aiming to output a segmentation mask based on complex and implicit textual queries about specific parts of a 3D object.
• To facilitate evaluation and benchmarking, we present a large 3D dataset comprising over 60k instructions paired with corresponding ground-truth part segmentation annotations specifically curated for reasoning-based 3D part segmentation.