The Unreasonable Effectiveness of the Baseline: Discussing SVMs in Legal Text Classification

Understand

We aim to highlight an interesting trend to contribute to the ongoing debate around advances within legal Natural Language Processing.

• Recently, the focus for most legal text classification tasks has shifted towards large pre-trained deep learning models such as BERT.
• In this paper, we show that a more traditional approach based on Support Vector Machine classifiers reaches surprisingly competitive performance with BERT-based models on the classification tasks in the LexGLUE benchmark.
• We also highlight that error reduction obtained by using specialised BERT-based models over baselines is noticeably smaller in the legal domain when compared to general language tasks.