Can Large Language Models Understand Preferences in Personalized Recommendation?

Understand

Large Language Models (LLMs) excel in various tasks, including personalized recommendations.

• Existing evaluation methods often focus on rating prediction, relying on regression errors between actual and predicted ratings.
• However, user rating bias and item quality, two influential factors behind rating scores, can obscure personal preferences in user-item pair data.
• To address this, we introduce PerRecBench, disassociating the evaluation from these two factors and assessing recommendation techniques on capturing the personal preferences in a grouped ranking manner.