AlephBERT:A Hebrew Large Pre-Trained Language Model to Start-off your Hebrew NLP Application With

Understand

Large Pre-trained Language Models (PLMs) have become ubiquitous in the development of language understanding technology and lie at the heart of many artificial intelligence advances.

• While advances reported for English using PLMs are unprecedented, reported advances using PLMs in Hebrew are few and far between.
• The problem is twofold.
• First, Hebrew resources available for training NLP models are not at the same order of magnitude as their English counterparts.