There's No Comparison: Reference-less Evaluation Metrics in Grammatical Error Correction

Understand

Current methods for automatically evaluating grammatical error correction (GEC) systems rely on gold-standard references.

• However, these methods suffer from penalizing grammatical edits that are correct but not in the gold standard.
• We show that reference-less grammaticality metrics correlate very strongly with human judgments and are competitive with the leading reference-based evaluation metrics.
• By interpolating both methods, we achieve state-of-the-art correlation with human judgments.