On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behavior

Understand

Human reading behavior is tuned to the statistics of natural language: the time it takes human subjects to read a word can be predicted from estimates of the word's probability in context.

• However, it remains an open question what computational architecture best characterizes the expectations deployed in real time by humans that determine the behavioral signatures of reading.
• Here we test over two dozen models, independently manipulating computational architecture and training dataset size, on how well their next-word expectations predict human reading time behavior on naturalistic text corpora.
• We find that across model architectures and training dataset sizes the relationship between word log-probability and reading time is (near-)linear.