Comparing Pre-trained Human Language Models: Is it Better with Human Context as Groups, Individual Traits, or Both?

Understand

Pre-trained language models consider the context of neighboring words and documents but lack any author context of the human generating the text.

• However, language depends on the author's states, traits, social, situational, and environmental attributes, collectively referred to as human context (Soni et al., 2024).
• Human-centered natural language processing requires incorporating human context into language models.
• Currently, two methods exist: pre-training with 1) group-wise attributes (e.g., over-45-year-olds) or 2) individual traits.