Learning a Continue-Thinking Token for Enhanced Test-Time Scaling

Understand

Test-time scaling has emerged as an effective approach for improving language model performance by utilizing additional compute at inference time.

• Recent studies have shown that overriding end-of-thinking tokens (e.g., replacing &#34;</think>&#34; with &#34;Wait&#34;) can extend reasoning steps and improve accuracy.
• In this work, we explore whether a dedicated continue-thinking token can be learned to trigger extended reasoning.
• We augment a distilled version of DeepSeek-R1 with a single learned &#34;<|continue-thinking|>&#34; token, training only its embedding via reinforcement learning while keeping the model weights frozen.