Omni TM-AE: A Scalable and Interpretable Embedding Model Using the Full Tsetlin Machine State Space

Understand

The increasing complexity of large-scale language models has amplified concerns regarding their interpretability and reusability.

• While traditional embedding models like Word2Vec and GloVe offer scalability, they lack transparency and often behave as black boxes.
• Conversely, interpretable models such as the Tsetlin Machine (TM) have shown promise in constructing explainable learning systems, though they previously faced limitations in scalability and reusability.
• In this paper, we introduce Omni Tsetlin Machine AutoEncoder (Omni TM-AE), a novel embedding model that fully exploits the information contained in the TM's state matrix, including literals previously excluded from clause formation.