VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning

Understand

Reinforcement Learning (RL) benefits Large Language Models (LLMs) for complex reasoning.

• Inspired by this, we explore integrating spatio-temporal specific rewards into Multimodal Large Language Models (MLLMs) to address the unique challenges of video understanding, such as long-range temporal associations.
• This paper investigates how rule-based rewards, particularly temporal ones, can improve video reasoning and their generalizability.
• Our study proposes Reinforcement Fine-Tuning (RFT) as a data-efficient method to enhance video reasoning on specific tasks without sacrificing original capabilities.