The relational processing limits of classic and contemporary neural network models of language processing

Understand

The ability of neural networks to capture relational knowledge is a matter of long-standing controversy.

• Recently, some researchers in the PDP side of the debate have argued that (1) classic PDP models can handle relational structure (Rogers & McClelland, 2008, 2014) and (2) the success of deep learning approaches to text processing suggests that structured representations are unnecessary to capture the gist of human language (Rabovsky et al., 2018).
• In the present study we tested the Story Gestalt model (St.
• John, 1992), a classic PDP model of text comprehension, and a Sequence-to-Sequence with Attention model (Bahdanau et al., 2015), a contemporary deep learning architecture for text processing.