Beyond Utility: Evaluating LLM as Recommender

Understand

With the rapid development of Large Language Models (LLMs), recent studies employed LLMs as recommenders to provide personalized information services for distinct users.

• Despite efforts to improve the accuracy of LLM-based recommendation models, relatively little attention is paid to beyond-utility dimensions.
• Moreover, there are unique evaluation aspects of LLM-based recommendation models, which have been largely ignored.
• To bridge this gap, we explore four new evaluation dimensions and propose a multidimensional evaluation framework.