Good, but not always Fair: An Evaluation of Gender Bias for three commercial Machine Translation Systems

Understand

Machine Translation (MT) continues to make significant strides in quality and is increasingly adopted on a larger scale.

• Consequently, analyses have been redirected to more nuanced aspects, intricate phenomena, as well as potential risks that may arise from the widespread use of MT tools.
• Along this line, this paper offers a meticulous assessment of three commercial MT systems - Google Translate, DeepL, and Modern MT - with a specific focus on gender translation and bias.
• For three language pairs (English/Spanish, English/Italian, and English/French), we scrutinize the behavior of such systems at several levels of granularity and on a variety of naturally occurring gender phenomena in translation.