Interactive Generation of Laparoscopic Videos with Diffusion Models

Understand

Generative AI, in general, and synthetic visual data generation, in specific, hold much promise for benefiting surgical training by providing photorealism to simulation environments.

• Current training methods primarily rely on reading materials and observing live surgeries, which can be time-consuming and impractical.
• In this work, we take a significant step towards improving the training process.
• Specifically, we use diffusion models in combination with a zero-shot video diffusion method to interactively generate realistic laparoscopic images and videos by specifying a surgical action through text and guiding the generation with tool positions through segmentation masks.