Update Frequently, Update Fast: Retraining Semantic Parsing Systems in a Fraction of Time

Understand

Currently used semantic parsing systems deployed in voice assistants can require weeks to train.

• Datasets for these models often receive small and frequent updates, data patches.
• Each patch requires training a new model.
• To reduce training time, one can fine-tune the previously trained model on each patch, but naive fine-tuning exhibits catastrophic forgetting - degradation of the model performance on the data not represented in the data patch.