Modulated noise reference unit (MNRU)

ITU-T Recommendation P.810, · 1996

Earlier work this paper cites.

Subjective test methodology for evaluating speech communication systems that include noise suppression algorithm

ITU-T Recommendation P.835, · 2003

Earlier work this paper cites.

“Development and validation of an automatic speech-in-noise screening test by telephone,”

Cas Smits, Theo S Kapteyn, and Tammo Houtgast, · 2004

Earlier work this paper cites.

“Subjective comparison and evaluation of speech enhancement algorithms,”

Yi Hu and Philipos C Loizou, · 2007

Earlier work this paper cites.

“Crowdmos: An approach for crowdsourcing mean opinion score studies,”

Flávio Ribeiro, Dinei Florêncio, Cha Zhang, and Michael Seltzer, · 2011

Earlier work this paper cites.

Better Reference System for the P.835 SIG Rating Scale

AH-11-029, · 2011

Earlier work this paper cites.

“Best practices and recommendations for crowdsourced qoe-lessons learned from the qualinet task force” crowdsourcing”,”

Tobias Hoßfeld, Matthias Hirth, Judith Redi, Filippo Mazza, Pavel Korshunov, Babak Naderi, Michael Seufert, Bruno Gardlo, Sebastian Egger, and Christian Keimel, · 2014

Earlier work this paper cites.

“Effect of Trapping Questions on the Reliability of Speech Quality Judgments in a Crowdsourcing Paradigm,”

Babak Naderi, Tim Polzehl, Ina Wechsung, Friedemann Köster, and Sebastian Möller, · 2015

Earlier work this paper cites.

Reference Impairments for Superwideband and Fullband P.835 Tests – Processing and Auditory Results

S4‑150762, · 2015

Earlier work this paper cites.

Revision of DESUDAPS-1: Common subjective testing framework for training and validation of SWB and FB P.835 test predictors v 1.2

S4‑160397, · 2016

Cited alongside, same era.

“Subjective speech quality measurement repeatability: comparison of laboratory test results,”

Jan Holub, Hakob Avetisyan, and Scott Isabelle, · 2017

Cited alongside, same era.

“Speech quality assessment in crowdsourcing: Influence of environmental noise,”

Babak Naderi, Sebastian Möller, and Gabriel Mittag, · 2018

Cited alongside, same era.

Subjective evaluation of speech quality with a crowdsourcing approach

ITU-T Recommendation P.808, · 2018

Cited alongside, same era.

Motivation of workers on microtask crowdsourcing platforms

Babak Naderi, · 2018

Cited alongside, same era.

Software tools for speech and audio coding standardization

“An open source implementation of itu-t recommendation p.808 with validation,”

Babak Naderi and Ross Cutler, · 2020

Closest in time.

“Application of Just-Noticeable Difference in Quality as Environment Suitability Test for Crowdsourcing Speech Quality Assessment Task,”

Babak Naderi and Sebastian Möller, · 2020

Closest in time.

“Towards speech quality assessment using a crowdsourcing approach: Evaluation of standardized methods,”

Babak Naderi, Rafael Zequeira Jiménez, Matthias Hirth, Sebastian Möller, Florian Metzger, and Tobias Hoßfeld, · 2020

Closest in time.

Test signals for use in telephony and other speech-based applications

ITU-T Recommendation P.501, · 2020

Closest in time.

“The interspeech 2020 deep noise suppression challenge: Datasets, subjective testing framework, and challenge results,”

Original

Chandan KA Reddy, Vishak Gopal, Ross Cutler, Ebrahim Beyrami, Roger Cheng, Harishchandra Dubey, Sergiy Matusevych, Robert Aichner, Ashkan Aazami, Sebastian Braun, et al., · 2020