A Generalized Language Model in Tensor Space

Understand

In the literature, tensors have been effectively used for capturing the context information in language models.

• However, the existing methods usually adopt relatively-low order tensors, which have limited expressive power in modeling language.
• Developing a higher-order tensor representation is challenging, in terms of deriving an effective solution and showing its generality.
• In this paper, we propose a language model named Tensor Space Language Model (TSLM), by utilizing tensor networks and tensor decomposition.