Sequential Recommendation with Relation-Aware Kernelized Self-Attention

Understand

Recent studies identified that sequential Recommendation is improved by the attention mechanism.

• By following this development, we propose Relation-Aware Kernelized Self-Attention (RKSA) adopting a self-attention mechanism of the Transformer with augmentation of a probabilistic model.
• The original self-attention of Transformer is a deterministic measure without relation-awareness.
• Therefore, we introduce a latent space to the self-attention, and the latent space models the recommendation context from relation as a multivariate skew-normal distribution with a kernelized covariance matrix from co-occurrences, item characteristics, and user information.