RPGBENCH: Evaluating Large Language Models as Role-Playing Game Engines

A very brief measure of the big-five personality domains

Gosling, S. D., Rentfrow, P. J., and Swann Jr, W. B · 2003

Earlier work this paper cites.

Introducing rpbench-auto, August 2024

BosonAI · 2024

Earlier work this paper cites.

Large language models know how the personality of public figures is perceived by the general public

Cao, X. and Kosinski, M · 2024

Earlier work this paper cites.

SocialBench: Sociality evaluation of role-playing conversational agents

Chen, H., Chen, H., Yan, M., Xu, W., Xing, G., Shen, W., Quan, X., Li, C., Zhang, J., and Huang, F · 2024

Earlier work this paper cites.

Mmrole: A comprehensive framework for developing and evaluating multimodal role-playing agents

Original

Dai, Y., Hu, H., Wang, L., Jin, S., Chen, X., and Lu, Z · 2024

Cited alongside, same era.

Pingpong: A benchmark for role-playing language models with user emulation and multi-model evaluation

Original

Gusev, I · 2024

Cited alongside, same era.

Personagym: Evaluating persona agents and llms

Original

Samuel, V., Zou, H. P., Zhou, Y., Chaudhari, S., Kalyan, A., Rajpurohit, T., Deshpande, A., Narasimhan, K., and Murahari, V · 2024

Cited alongside, same era.

From persona to personalization: A survey on role-playing language agents

Chen, J., Wang, X., Xu, R., Yuan, S., Zhang, Y., Shi, W., Xie, J., Li, S., Yang, R., Zhu, T., Chen, A., Li, N., Chen, L., Hu, C., Wu, S., Ren, S., Fu, Z., and Xiao, Y

Cited in the paper.

Characterbox: Evaluating the role-playing capabilities of llms in text-based virtual worlds

Original

Wang, L., Lian, J., Huang, Y., Dai, Y., Li, H., Chen, X., Xie, X., and Wen, J.-R

Cited in the paper.

CharacterEval: A Chinese benchmark for role-playing conversational agent evaluation

Tu, Q., Fan, S., Tian, Z., Shen, T., Shang, S., Gao, X., and Yan, R · 2024

Later among the works it cites.

InCharacter: Evaluating personality fidelity in role-playing agents through psychological interviews

Wang, X., Xiao, Y., Huang, J.-t., Yuan, S., Xu, R., Guo, H., Tu, Q., Fei, Y., Leng, Z., Wang, W., Chen, J., Li, C., and Xiao, Y · 2024

Later among the works it cites.

Evaluating character understanding of large language models via character profiling from fictional works

Original

Yuan, X., Yuan, S., Cui, Y., Lin, T., Wang, X., Xu, R., Chen, J., and Yang, D · 2024

Later among the works it cites.