Zero Token-Driven Deep Thinking in LLMs: Unlocking the Full Potential of Existing Parameters via Cyclic Refinement

Understand

Resource limitations often constrain the parameter counts of Large Language Models (LLMs), hindering their performance.

• While existing methods employ parameter sharing to reuse the same parameter set under fixed budgets, such approaches typically force each layer to assume multiple roles with a predetermined number of iterations, restricting efficiency and adaptability.
• In this work, we propose the Zero Token Transformer (ZTT), which features a head-tail decoupled parameter cycling method.
• We disentangle the first (head) and last (tail) layers from parameter cycling and iteratively refine only the intermediate layers.