Lattice: Learning to Efficiently Compress the Memory

Understand

Attention mechanisms have revolutionized sequence learning but suffer from quadratic computational complexity.

• This paper introduces \model, a novel recurrent neural network (RNN) mechanism that leverages the inherent low-rank structure of K-V matrices to efficiently compress the cache into a fixed number of memory slots, achieving sub-quadratic complexity.
• We formulate this compression as an online optimization problem and derive a dynamic memory update rule based on a single gradient descent step.
• The resulting recurrence features a state- and input-dependent gating mechanism, offering an interpretable memory update process.