State-offset Tuning: State-based Parameter-Efficient Fine-Tuning for State Space Models

Understand

State Space Models (SSMs) have emerged as efficient alternatives to Transformers, mitigating their quadratic computational cost.

• However, the application of Parameter-Efficient Fine-Tuning (PEFT) methods to SSMs remains largely unexplored.
• In particular, prompt-based methods like Prompt Tuning and Prefix-Tuning, which are widely used in Transformers, do not perform well on SSMs.
• To address this, we propose state-based methods as a superior alternative to prompt-based methods.