PIM-GPT: A Hybrid Process-in-Memory Accelerator for Autoregressive Transformers

Understand

Decoder-only Transformer models such as GPT have demonstrated exceptional performance in text generation, by autoregressively predicting the next token.

• However, the efficacy of running GPT on current hardware systems is bounded by low compute-to-memory-ratio and high memory access.
• Process-in-memory (PIM) architectures can minimize off-chip data movement and utilize high internal bandwidth.
• They stand out as promising candidates for accelerating memory-bounded tasks such as GPT inference.