Investigating Range-Equalizing Bias in Mean Opinion Score Ratings of Synthesized Speech

Understand

Mean Opinion Score (MOS) is a popular measure for evaluating synthesized speech.

• However, the scores obtained in MOS tests are heavily dependent upon many contextual factors.
• One such factor is the overall range of quality of the samples presented in the test -- listeners tend to try to use the entire range of scoring options available to them regardless of this, a phenomenon which is known as range-equalizing bias.
• In this paper, we systematically investigate the effects of range-equalizing bias on MOS tests for synthesized speech by conducting a series of listening tests in which we progressively "zoom in" on a smaller number of systems in the higher-quality range.