GLSFormer: Gated - Long, Short Sequence Transformer for Step Recognition in Surgical Videos

Understand

Automated surgical step recognition is an important task that can significantly improve patient safety and decision-making during surgeries.

• Existing state-of-the-art methods for surgical step recognition either rely on separate, multi-stage modeling of spatial and temporal information or operate on short-range temporal resolution when learned jointly.
• However, the benefits of joint modeling of spatio-temporal features and long-range information are not taken in account.
• In this paper, we propose a vision transformer-based approach to jointly learn spatio-temporal features directly from sequence of frame-level patches.