Compositional Distributional Semantics with Long Short Term Memory

Understand

We are proposing an extension of the recursive neural network that makes use of a variant of the long short-term memory architecture.

• The extension allows information low in parse trees to be stored in a memory register (the `memory cell') and used much later higher up in the parse tree.
• This provides a solution to the vanishing gradient problem and allows the network to capture long range dependencies.
• Experimental results show that our composition outperformed the traditional neural-network composition on the Stanford Sentiment Treebank.