Fetching the paper…
Reading the bibliography…
We conducted an experiment during the review process of the 2023 International Conference on Machine Learning (ICML), asking authors with multiple submissions to rank their papers based on perceived quality.
The isotonic regression problem and its dual
R. E. Barlow and H. D. Brunk · 1972
Earlier work this paper cites.
Peering at the peer review process for conference submissions
A. Gardner, K. Willey, L. Jolly, and G. Tibbits · 2012
Earlier work this paper cites.
The arbitrariness of reviews, and advice for school administrators
J. Langford and M. Guzdial · 2015
Earlier work this paper cites.
Cvpr paper controversy; ml community reviews peer review
T. Peng · 2018
Earlier work this paper cites.
Design and analysis of the NIPS 2016 review process
N. Shah, B. Tabibian, K. Muandet, I. Guyon, and U. Von Luxburg · 2018
Earlier work this paper cites.
Strategyproof peer selection using randomization, partitioning, and apportionment
H. Aziz, O. Lev, N. Mattei, J. S. Rosenschein, and T. Walsh · 2019
Earlier work this paper cites.
Paper matching with local fairness constraints
A. Kobren, B. Saha, and A. McCallum · 2019
Earlier work this paper cites.
Troubling trends in machine learning scholarship: Some ml papers suffer from flaws that could mislead the public and stymie future research
Z. C. Lipton and J. Steinhardt · 2019
Earlier work this paper cites.
Avoiding a tragedy of the commons in the peer review process
D. Sculley, J. Snoek, and A. Wiltschko · 2019
Earlier work this paper cites.
Your 2 is my 1, your 3 is my 9: Handling arbitrary miscalibrations in ratings
J. Wang and N. Shah · 2019
Earlier work this paper cites.
Arbitrariness in the peer review process
E. S. Brezis and A. Birukou · 2020
Earlier work this paper cites.
Mitigating manipulation in peer review via randomized reviewer assignments
S. Jecmen, H. Zhang, R. Liu, N. Shah, V. Conitzer, and F. Fang · 2020
Cited alongside, same era.
Peernomination: Relaxing exactness for increased accuracy in peer selection
N. Mattei, P. Turrini, and S. Zhydkov · 2020
Cited alongside, same era.
ICLR2020-openreviewdata
S.-H. Sun · 2020
Cited alongside, same era.
Debiasing evaluations that are biased by evaluations
J. Wang, I. Stelmakh, Y. Wei, and N. B. Shah · 2020
Cited alongside, same era.
Inconsistency in conference peer review: Revisiting the 2014 NeurIPS experiment
C. Cortes and N. D. Lawrence · 2021
Cited alongside, same era.
Should peer reviewers be paid to review academic papers?
P. Y. Cheah and J. Piasecki · 2022
Later among the works it cites.
C. Rastogi, I. Stelmakh, A. Beygelzimer, Y. N. Dauphin, P. Liang, J. W. Vaughan, Z. Xue, H. Daumé III, E. Pierson, and N. B. Shah · 2022
Later among the works it cites.
A truthful owner-assisted scoring mechanism
W. J. Su · 2022
Later among the works it cites.
Can we automate scientific reviewing?
W. Yuan, P. Liu, and G. Neubig · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Is the quality of reviews reflected in editors’ and authors’ satisfaction with peer review? a cross-sectional study in 12 journals across four research fields
S. M. Pranić, M. Malicki, S. L. Marusić, B. Mehmani, and A. Marusić · 2021
Cited alongside, same era.
Some ethical issues in the review process of machine learning conferences
A. Russo · 2021
Cited alongside, same era.
Auctions and prediction markets for scientific peer review
S. Srinivasan and J. Morgenstern · 2021
Cited alongside, same era.
A novice-reviewer experiment to address scarcity of qualified reviewers in large conferences
I. Stelmakh, N. B. Shah, A. Singh, and H. Daumé III · 2021
Cited alongside, same era.
You are the best reviewer of your own papers: An owner-assisted scoring mechanism
W. J. Su · 2021
Cited alongside, same era.
No free lunch in “privacy for free: How does dataset condensation help privacy”
N. Carlini, V. Feldman, and M. Nasr · 2022
Cited alongside, same era.
Ranking inferences based on the top choice of multiway comparisons
J. Fan, Z. Lou, W. Wang, and M. Yu
Cited in the paper.
A. Beygelzimer, Y. N. Dauphin, P. Liang, and J. W. Vaughan · 2023
Later among the works it cites.
How random is the review outcome? a systematic study of the impact of external factors on elife peer review
W. Liang, K. Mahowald, J. Raymond, V. Krishna, D. Smith, D. Jurafsky, D. McFarland, and J. Zou · 2023
Later among the works it cites.
Yet another ICML award fiasco
F. Orabona · 2023
Later among the works it cites.
Peer prediction for peer review: designing a marketplace for ideas
A. Ugarov · 2023
Later among the works it cites.
A truth serum for eliciting self-evaluations in scientific reviews
J. Wu, H. Xu, Y. Guo, and W. J. Su · 2023
Later among the works it cites.
Peer reviews of peer reviews: A randomized controlled trial and other experiments
A. Goldberg, I. Stelmakh, K. Cho, A. Oh, A. Agarwal, D. Belgrave, and N. B. Shah · 2025
Closest in time.
Isotonic mechanism for exponential family estimation in machine learning peer review
Y. Yan, W. J. Su, and J. Fan · 2025
Closest in time.