SophiaVL-R1: Reinforcing MLLMs Reasoning with Thinking Reward

Understand

Recent advances have shown success in eliciting strong reasoning abilities in multimodal large language models (MLLMs) through rule-based reinforcement learning (RL) with outcome rewards.

• However, this paradigm typically lacks supervision over the thinking process leading to the final outcome.
• As a result, the model may learn sub-optimal reasoning strategies, which can hinder its generalization ability.
• In light of this, we propose SophiaVL-R1, as an attempt to add reward signals for the thinking process in this paradigm.