ParsBERT: Transformer-based Model for Persian Language Understanding

Understand

The surge of pre-trained language models has begun a new era in the field of Natural Language Processing (NLP) by allowing us to build powerful language models.

• Among these models, Transformer-based models such as BERT have become increasingly popular due to their state-of-the-art performance.
• However, these models are usually focused on English, leaving other languages to multilingual models with limited resources.
• This paper proposes a monolingual BERT for the Persian language (ParsBERT), which shows its state-of-the-art performance compared to other architectures and multilingual models.