Fetching the paper…
Reading the bibliography…
Reproducibility is an ideal that no researcher would dispute "in the abstract", but when aspirations meet the cold hard reality of the academic grind, reproducibility often "loses out".
The Need for Open Source Software in Machine Learning
Sören Sonnenburg, Mikio L. Braun, Cheng Soon Ong, Samy Bengio, Leon Bottou, Geoffrey Holmes, Yann LeCun, Klaus-Robert Müller, Fernando Pereira, Carl Edward Rasmussen, Gunnar Rätsch, Bernhard Schölkopf, Alexander Smola, Pascal Vincent, Jason Weston, and Robert Williamson. 2007 · 2007
Earlier work this paper cites.
Replicability is not Reproducibility: Nor is it Good Science. In Proceedings of the 4th Workshop on Evaluation Methods for Machine Learning at ICML . Montréal, Québec, Canada
Chris Drummond. 2009 · 2009
Earlier work this paper cites.
Promoting Repeatability Through Open Runs. In Proceedings of the 7th International Workshop on Evaluating Information Access (EVIA 2016) . Tokyo, Japan, 17–20
Ellen M. Voorhees, Shahzad Rajput, and Ian Soboroff. 2016 · 2016
Earlier work this paper cites.
The FAIR Guiding Principles for Scientific Data Management and Stewardship
Mark D. Wilkinson, Michel Dumontier, IJsbrand Jan Aalbersberg, Gabrielle Appleton, Myles Axton, Arie Baak, Niklas Blomberg, Jan-Willem Boiten, Luiz Bonino da Silva Santos, Philip E. Bourne, Jildau Bouwman, Anthony J. Brookes, Tim Clark, Mercè Crosas, Ingrid Dillo, Olivier Dumon, Scott Edmunds, Chris T. Evelo, Richard Finkers, Alejandra Gonzalez-Beltran, Alasdair J.G. Gray, Paul Groth, Carole Goble, Jeffrey S. Grethe abd Jaap Heringa, Peter A.C. ’t Hoen, Rob Hooft, Tobias Kuhn, Ruben Kok, Joost Kok, Scott J. Lusher, Maryann E. Martone, Albert Mons, Abel L. Packer, Bengt Persson, Philippe Rocca-Serra, Marco Roos, Rene van Schaik, Susanna-Assunta Sansone, Erik Schultes, Thierry Sengstag, Ted Slater, George Strawn, Morris A. Swertz, Mark Thompson, Johan van der Lei, Erik van Mulligen, Jan Velterop, Andra Waagmeester, Peter Wittenburg, Katherine Wolstencroft, Jun Zhao, and Barend Mons. 2016 · 2016
Earlier work this paper cites.
Anserini: Enabling the Use of Lucene for Information Retrieval Research. In Proceedings of the 40th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2017) . Tokyo, Japan, 1253–1256
Peilin Yang, Hui Fang, and Jimmy Lin. 2017 · 2017
Earlier work this paper cites.
MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang. 2018 · 2018
Earlier work this paper cites.
Anserini: Reproducible Ranking Baselines Using Lucene
Peilin Yang, Hui Fang, and Jimmy Lin. 2018 · 2018
Earlier work this paper cites.
A Large-scale Study about Quality and Reproducibility of Jupyter Notebooks. In Proceedings of the 16th International Conference on Mining Software Repositories (MSR 2019) . Montréal, Québec, Canada
João Felipe Pimentel, Leonardo Murta, Vanessa Braganholo, and Juliana Freire. 2019 · 2019
Cited alongside, same era.
Dense Passage Retrieval for Open-Domain Question Answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) . Online, 6769–6781
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020 · 2020
Cited alongside, same era.
Towards FAIR Principles for Research Software
Anna-Lena Lamprecht, Leyla Garcia, Mateusz Kuzak, Carlos Martinez, Ricardo Arcila, Eva Martin Del Pico, Victoria Dominguez Del Angel, Stephanie van de Sandt, Jon Ison, Paula Andrea Martinez, Peter McQuilton, Alfonso Valencia, Jennifer Harrow, Fotis Psomopoulos, Josep Ll. Gelpi, Neil Chue Hong, Carole Goble, and Salvador Capella-Gutierrez. 2020 · 2020
Cited alongside, same era.
Reproducibility is a Process, not an Achievement: The Replicability of IR Reproducibility Experiments. In Proceedings of the 42nd European Conference on Information Retrieval, Part II (ECIR 2020) . 43–49
Jimmy Lin and Qian Zhang. 2020 · 2020
Jimmy Lin and Xueguang Ma. 2021 · 2021
Later among the works it cites.
Pyserini: A Python Toolkit for Reproducible Information Retrieval Research with Sparse and Dense Representations. In Proceedings of the 44th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2021) . 2356–2362
Jimmy Lin, Xueguang Ma, Sheng-Chieh Lin, Jheng-Hong Yang, Ronak Pradeep, and Rodrigo Nogueira. 2021a · 2021
Later among the works it cites.
The Missing README: A Guide for the New Software Engineer
Chris Riccomini and Dmitriy Ryaboy. 2021 · 2021
Later among the works it cites.
Introducing the FAIR Principles for Research Software
Michelle Barker, Neil P. Chue Hong, Daniel S. Katz, Anna-Lena Lamprecht, Carlos Martinez-Ortiz, Fotis Psomopoulos, Jennifer Harrow, Leyla Jael Castro, Morane Gruenpeter, Paula Andrea Martinez, and Tom Honeyman. 2022 · 2022
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
TREC-COVID: Rationale and Structure of an Information Retrieval Shared Task for COVID-19
Kirk Roberts, Tasmeer Alam, Steven Bedrick, Dina Demner-Fushman, Kyle Lo, Ian Soboroff, Ellen Voorhees, Lucy Lu Wang, and William R. Hersh. 2020 · 2020
Cited alongside, same era.
Overview of the TREC 2021 Deep Learning Track. In Proceedings of the Thirtieth Text REtrieval Conference (TREC 2021) . Gaithersburg, Maryland
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Jimmy Lin. 2021 · 2021
Cited alongside, same era.
A Proposed Conceptual Framework for a Representational Approach to Information Retrieval
Jimmy Lin. 2021 · 2021
Cited alongside, same era.
Pretrained Transformers for Text Ranking: BERT and Beyond
Jimmy Lin, Rodrigo Nogueira, and Andrew Yates. 2021b
Cited in the paper.
Aligning the Research and Practice of Building Search Applications: Elasticsearch and Pyserini. In Proceedings of the 15th ACM International Conference on Web Search and Data Mining (WSDM 2022) . 1573–1576
Josh Devins, Julie Tibshirani, and Jimmy Lin. 2022 · 2022
Closest in time.
Document Expansions and Learned Sparse Lexical Representations for MS MARCO V1 and V2. In Proceedings of the 45th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2022) . Madrid, Spain, 3187–3197
Xueguang Ma, Ronak Pradeep, Rodrigo Nogueira, and Jimmy Lin. 2022a · 2022
Closest in time.
Another Look at DPR: Reproduction of Training and Replication of Retrieval. In Proceedings of the 44th European Conference on Information Retrieval (ECIR 2022), Part I . Stavanger, Norway, 613–626
Xueguang Ma, Kai Sun, Ronak Pradeep, Minghan Li, and Jimmy Lin. 2022b · 2022
Closest in time.