Position: The Artificial Intelligence and Machine Learning Community Should Adopt a More Transparent and Regulated Peer Review Process

The pagerank citation ranking: Bringing order to the web

Page, L · 1999

Earlier work this paper cites.

Crafting papers on machine learning

Langley, P · 2000

Earlier work this paper cites.

Confidentiality and peer review: The paradox of secrecy in academe

Baez, B · 2002

Earlier work this paper cites.

What is open peer review? a systematic review

Ross-Hellauer, T · 2017

Earlier work this paper cites.

Privacy versus open science

Dennis, S., Garrett, P., Yim, H., Hamm, J., Osth, A. F., Sreekumar, V., and Stone, B · 2019

Earlier work this paper cites.

Guidelines for open peer review implementation

Ross-Hellauer, T. and Görögh, E · 2019

Earlier work this paper cites.

Open peer review: promoting transparency in open science

Wolfram, D., Wang, P., Hembree, A., and Park, H · 2020

Earlier work this paper cites.

Inconsistency in conference peer review: Revisiting the 2014 neurips experiment

Original

Cortes, C. and Lawrence, N. D · 2021

Earlier work this paper cites.

An open review of openreview: A critical analysis of the machine learning conference review process, 2021

Tran, D., Valtchanov, A. V., Ganapathy, K. R., Feng, R., Slud, E. V., Goldblum, M., and Goldstein, T · 2021

Earlier work this paper cites.

How confident was your reviewer? estimating reviewer confidence from peer review texts

Bharti, P. K., Ghosal, T., Agrawal, M., and Ekbal, A · 2022

Earlier work this paper cites.

Can transparency undermine peer review? a simulation model of scientist behavior under open peer review

Bianchi, F. and Squazzoni, F · 2022

Earlier work this paper cites.

Innovating peer review, reconfiguring scholarly communication: An analytical overview of ongoing peer review innovation activities

Kaltenbrunner, W., Pinfield, S., Waltman, L., Woods, H. B., and Brumberg, J · 2022

Earlier work this paper cites.

The neurips experiment, May 2022

Lawrence, N. D · 2022

Cited alongside, same era.

Challenges, experiments, and computational solutions in peer review

Shah, N. B · 2022

Cited alongside, same era.

Managing the whole research process on github

Takagi, S · 2022

Cited alongside, same era.

Investigating fairness disparities in peer review: A language model enhanced approach

Original

Zhang, J., Zhang, H., Deng, Z., and Roth, D · 2022

Cited alongside, same era.

Has the machine learning review process become more arbitrary as the field has grown? the neurips 2021 consistency experiment

Original

Beygelzimer, A., Dauphin, Y. N., Liang, P., and Vaughan, J. W · 2023

Cited alongside, same era.

The review mills, not just (self-) plagiarism in review reports, but a step further

Oviedo-García, M. Á · 2024

Later among the works it cites.

Emerging plagiarism in peer-review evaluation reports: a tip of the iceberg?

Piniewski, M., Jarić, I., Koutsoyiannis, D., and Kundzewicz, Z. W · 2024

Later among the works it cites.

Ai-powered peer review needs human supervision

Seghier, M. L · 2024

Later among the works it cites.

Analysis of the icml 2023 ranking data: Can authors’ opinions of their own papers assist peer review in machine learning?

Original

Su, B., Zhang, J., Collina, N., Yan, Y., Li, D., Cho, K., Fan, J., Roth, A., and Su, W. J · 2024

Later among the works it cites.

Position: AI/ML influencers have a place in the academic process

Weissburg, I., Arora, M., Wang, X., Pan, L., and Wang, W. Y · 2024

Later among the works it cites.