Synthetic Data Generation for Grammatical Error Correction with Tagged Corruption Models

Understand

Synthetic data generation is widely known to boost the accuracy of neural grammatical error correction (GEC) systems, but existing methods often lack diversity or are too simplistic to generate the broad range of grammatical errors made by human writers.

• In this work, we use error type tags from automatic annotation tools such as ERRANT to guide synthetic data generation.
• We compare several models that can produce an ungrammatical sentence given a clean sentence and an error type tag.
• We use these models to build a new, large synthetic pre-training data set with error tag frequency distributions matching a given development set.