The Elephant in the Room: Rethinking the Usage of Pre-trained Language Model in Sequential Recommendation

Understand

Sequential recommendation (SR) has seen significant advancements with the help of Pre-trained Language Models (PLMs).

• Some PLM-based SR models directly use PLM to encode user historical behavior's text sequences to learn user representations, while there is seldom an in-depth exploration of the capability and suitability of PLM in behavior sequence modeling.
• In this work, we first conduct extensive model analyses between PLMs and PLM-based SR models, discovering great underutilization and parameter redundancy of PLMs in behavior sequence modeling.
• Inspired by this, we explore different lightweight usages of PLMs in SR, aiming to maximally stimulate the ability of PLMs for SR while satisfying the efficiency and usability demands of practical systems.