Can Large Language Models Understand Context?

Understand

Understanding context is key to understanding human language, an ability which Large Language Models (LLMs) have been increasingly seen to demonstrate to an impressive extent.

• However, though the evaluation of LLMs encompasses various domains within the realm of Natural Language Processing, limited attention has been paid to probing their linguistic capability of understanding contextual features.
• This paper introduces a context understanding benchmark by adapting existing datasets to suit the evaluation of generative models.
• This benchmark comprises of four distinct tasks and nine datasets, all featuring prompts designed to assess the models' ability to understand context.