Fetching the paper…

Reading the bibliography…

2017

Effectiveness of Anonymization in Double-Blind Review

Goues, Claire Le, Brun, Yuriy, Apel, Sven et al.

Original
PDF
Post on X

Understand

Double-blind review relies on the authors' ability and willingness to effectively anonymize their submissions.

We explore anonymization effectiveness at ASE 2016, OOPSLA 2016, and PLDI 2016 by asking reviewers if they can guess author identities.
We find that 74%-90% of reviews contain no correct guess and that reviewers who self-identify as experts on a paper's topic are more likely to attempt to guess, but no more likely to guess correctly.
We present our findings, summarize the PC chairs' comments about administering double-blind review, discuss the advantages and disadvantages of revealing author identities part of the way through the process, and conclude by advocating for the continued use of double-blind review.

Effectiveness of Anonymization in Double-Blind Review · Around

Built on

US and non-US submissions: An analysis of reviewer bias
Gastroenterology, Bethesda, MD, USA · 1998
Earlier work this paper cites.
Single- versus double-blind reviewing: An analysis of the literature
Richard Snodgrass · 2006
Earlier work this paper cites.

Similar

Double-blind review favours increased representation of female authors
Amber E. Budden, Tom Tregenza, Lonnie W. Aarssen, Julia Koricheva, Roosa Leimu, and Christopher J. Lortie · 2008
Cited alongside, same era.
Science faculty’s subtle gender biases favor male students
Corinne A. Moss-Racusin, John F. Dovidio, Victoria L. Brescoll, Mark J. Graham, and Jo Handelsman · 2012
Cited alongside, same era.

Then

Double-blind reviewing at EvoLang 11 reveals gender bias
Seán G. Roberts and Tessa Verhoef · 2016
Later among the works it cites.
Single versus double blind reviewing at WSDM 2017
Original
Andrew Tomkins, Min Zhang, and William D. Heavlin · 2017
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…