MultiLegalPile: A 689GB Multilingual Legal Corpus

Understand

Large, high-quality datasets are crucial for training Large Language Models (LLMs).

• However, so far, there are few datasets available for specialized critical domains such as law and the available ones are often only for the English language.
• We curate and release MultiLegalPile, a 689GB corpus in 24 languages from 17 jurisdictions.
• The MultiLegalPile corpus, which includes diverse legal data sources with varying licenses, allows for pretraining NLP models under fair use, with more permissive licenses for the Eurlex Resources and Legal mC4 subsets.