PTUM: Pre-training User Model from Unlabeled User Behaviors via Self-supervision

Understand

User modeling is critical for many personalized web services.

• Many existing methods model users based on their behaviors and the labeled data of target tasks.
• However, these methods cannot exploit useful information in unlabeled user behavior data, and their performance may be not optimal when labeled data is scarce.
• Motivated by pre-trained language models which are pre-trained on large-scale unlabeled corpus to empower many downstream tasks, in this paper we propose to pre-train user models from large-scale unlabeled user behaviors data.