Enabling Waypoint Generation for Collaborative Robots using LLMs and Mixed Reality

Understand

Programming a robotic is a complex task, as it demands the user to have a good command of specific programming languages and awareness of the robot's physical constraints.

• We propose a framework that simplifies robot deployment by allowing direct communication using natural language.
• It uses large language models (LLM) for prompt processing, workspace understanding, and waypoint generation.
• It also employs Augmented Reality (AR) to provide visual feedback of the planned outcome.