LEGAL-BERT: The Muppets straight out of Law School

Understand

BERT has achieved impressive performance in several NLP tasks.

• However, there has been limited investigation on its adaptation guidelines in specialised domains.
• Here we focus on the legal domain, where we explore several approaches for applying BERT models to downstream legal tasks, evaluating on multiple datasets.
• Our findings indicate that the previous guidelines for pre-training and fine-tuning, often blindly followed, do not always generalize well in the legal domain.