RoboNurse-VLA: Robotic Scrub Nurse System based on Vision-Language-Action Model

Understand

In modern healthcare, the demand for autonomous robotic assistants has grown significantly, particularly in the operating room, where surgical tasks require precision and reliability.

• Robotic scrub nurses have emerged as a promising solution to improve efficiency and reduce human error during surgery.
• However, challenges remain in terms of accurately grasping and handing over surgical instruments, especially when dealing with complex or difficult objects in dynamic environments.
• In this work, we introduce a novel robotic scrub nurse system, RoboNurse-VLA, built on a Vision-Language-Action (VLA) model by integrating the Segment Anything Model 2 (SAM 2) and the Llama 2 language model.