Provable unlearning in topic modeling and downstream tasks

Understand

Machine unlearning algorithms are increasingly important as legal concerns arise around the provenance of training data, but verifying the success of unlearning is often difficult.

• Provable guarantees for unlearning are often limited to supervised learning settings.
• In this paper, we provide the first theoretical guarantees for unlearning in the pre-training and fine-tuning paradigm by studying topic models, simple bag-of-words language models that can be adapted to solve downstream tasks like retrieval and classification.
• First, we design a provably effective unlearning algorithm for topic models that incurs a computational overhead independent of the size of the original dataset.