Unified Detoxifying and Debiasing in Language Generation via Inference-time Adaptive Optimization

Understand

Warning: this paper contains model outputs exhibiting offensiveness and biases.

• Recently pre-trained language models (PLMs) have prospered in various natural language generation (NLG) tasks due to their ability to generate fairly fluent text.
• Nevertheless, these models are observed to capture and reproduce harmful contents in training corpora, typically toxic language and social biases, raising severe moral issues.
• Prior works on ethical NLG tackle detoxifying and debiasing separately, which is problematic since we find debiased models still exhibit toxicity while detoxified ones even exacerbate social biases.