Fetching the paper…
Reading the bibliography…
Research is facing a reproducibility crisis, in which the results and findings of many studies are difficult or even impossible to reproduce.
1903
Earlier work this paper cites.
1907
Earlier work this paper cites.
Goecks, J., Nekrutenko, A., Taylor, J., The Galaxy Team: Galaxy: a comprehensive approach for supporting accessible, reproducible, and transparent computational research in the life sciences. Genome Biology 11
2010
Earlier work this paper cites.
Hong, S.Y., Koo, M.S., Jang, J., Kim, J.E.E., Park, H., Joh, M.S., Kang, J.H., Oh, T.J.: An Evaluation of the Software System Dependency of a Global Atmospheric Model. Monthly Weather Review 141
2013
Earlier work this paper cites.
Boettiger, C.: An introduction to Docker for reproducible research. ACM SIGOPS Operating Systems Review 49
2015
Earlier work this paper cites.
Peng, R.: The reproducibility crisis in science: A statistical counterattack. Significance 12
2015
Earlier work this paper cites.
Baker, M.: 1,500 scientists lift the lid on reproducibility. Nature 533
2016
Earlier work this paper cites.
Beel, J., Breitinger, C., Langer, S., Lommatzsch, A., Gipp, B.: Towards reproducibility in recommender-systems research. User Modeling and User-Adapted Interaction 26
2016
Earlier work this paper cites.
2016
Earlier work this paper cites.
Dror, R., Baumer, G., Bogomolov, M., Reichart, R.: Replicability Analysis for Natural Language Processing: Testing Significance with Multiple Datasets. Transactions of the Association for Computational Linguistics 5
2017
Earlier work this paper cites.
Rougier, N.P., Hinsen, K., Alexandre, F., Arildsen, T., Barba, L.A., Benureau, F.C.Y., Brown, C.T., Buyl, P.d., Caglayan, O., Davison, A.P., Delsuc, M.A., Detorakis, G., Diem, A.K., Drix, D., Enel, P., Girard, B., Guest, O., Hall, M.G., Henriques, R.N., Hinaut, X., Jaron, K.S., Khamassi, M., Klein, A., Manninen, T., Marchesi, P., McGlinn, D., Metzner, C., Petchey, O., Plesser, H.E., Poisot, T., Ram, K., Ram, Y., Roesch, E., Rossant, C., Rostami, V., Shifman, A., Stachelek, J., Stimberg, M., Stollmeier, F., Vaggi, F., Viejo, G., Vitay, J., Vostinar, A.E., Yurchak, R., Zito, T.: Sustainable computational science: the ReScience initiative. PeerJ Computer Science 3
2017
Earlier work this paper cites.
Clyburne-Sherin, A., Fei, X., Green, S.A.: Computational Reproducibility via Containers in Psychology. Meta-Psychology 3
2018
Earlier work this paper cites.
Gundersen, O.E., Kjensmo, S.: State of the Art: Reproducibility in Artificial Intelligence. Proceedings of the AAAI Conference on Artificial Intelligence 32
2018
Earlier work this paper cites.
Hardwicke, T.E., Mathur, M.B., MacDonald, K., Nilsonne, G., Banks, G.C., Kidwell, M.C., Hofelich Mohr, A., Clayton, E., Yoon, E.J., Henry Tessler, M., Lenne, R.L., Altman, S., Long, B., Frank, M.C.: Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition. Royal Society Open Science 5
2018
Earlier work this paper cites.
Hutson, M.: Artificial intelligence faces reproducibility crisis Unpublished code and sensitivity to training conditions make many claims hard to verify. Science 359
2018
Earlier work this paper cites.
Hutson, M.: Missing data hinder replication of artificial intelligence studies. Science (2018). https://doi.org/10.1126/science.aat3298
2018
Earlier work this paper cites.
Khetarpal, K., Ahmed, Z., Cianflone, A., Islam, R., Pineau, J.: RE-EVALUATE: Reproducibility in Evaluating Reinforcement Learning Algorithms. ICML (2018), https://openreview.net/forum?id=HJgAmITcgm
2018
Cited alongside, same era.
Nagarajan, P., Warnell, G., Stone, P.: The Impact of Nondeterminism on Reproducibility in Deep Reinforcement Learning. ICML (2018), https://openreview.net/forum?id=S1e-OsZ4e7
2018
Cited alongside, same era.
Sayre, F., Riegelman, A.: The Reproducibility Crisis and Academic Libraries | Sayre | College & Research Libraries. College & Research Libraries (2018). https://doi.org/https://doi.org/10.5860/crl.79.1.2, https://journals.acrl.org/index.php/crl/article/view/16846
2018
Cited alongside, same era.
Abel, D.: simple rl: Reproducible Reinforcement Learning in Python. ICLR (2019)
2019
Cited alongside, same era.
Pouchard, L., Lin, Y., Van Dam, H.: Replicating Machine Learning Experiments in Materials Science. In: Parallel Computing: Technology Trends, pp. 743–755. IOS Press (2020). https://doi.org/10.3233/APC200105, https://ebooks.iospress.nl/doi/10.3233/APC200105
2020
Later among the works it cites.
Artrith, N., Butler, K.T., Coudert, F.X., Han, S., Isayev, O., Jain, A., Walsh, A.: Best practices in machine learning for chemistry. Nature Chemistry 13
2021
Later among the works it cites.
2021
Later among the works it cites.
Cremonesi, P., Jannach, D.: Progress in Recommender Systems Research: Crisis? What Crisis? AI Magazine 42
2021
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
2019
Cited alongside, same era.
Gibney, E.: This AI researcher is trying to ward off a reproducibility crisis. Nature 577
2019
Cited alongside, same era.
Gundersen, O.E.: Standing on the Feet of Giants — Reproducibility in AI. AI Magazine 40
2019
Cited alongside, same era.
2019
Cited alongside, same era.
2019
Cited alongside, same era.
Nosek, B.A., Beck, E.D., Campbell, L., Flake, J.K., Hardwicke, T.E., Mellor, D.T., van ’t Veer, A.E., Vazire, S.: Preregistration Is Hard, And Worthwhile. Trends in Cognitive Sciences 23
2019
Cited alongside, same era.
Piantadosi, G., Marrone, S., Sansone, C.: On Reproducibility of Deep Convolutional Neural Networks Approaches. In: Kerautret, B., Colom, M., Lopresti, D., Monasse, P., Talbot, H. (eds.) Reproducible Research in Pattern Recognition. pp. 104–109. Lecture Notes in Computer Science, Springer International Publishing, Cham (2019). https://doi.org/10.1007/978-3-030-23987-9_10
2019
Cited alongside, same era.
Strømland, E.: Preregistration and reproducibility. Journal of Economic Psychology 75
2019
Cited alongside, same era.
Gundersen, O.E., Shamsaliei, S., Isdahl, R.J.: Do machine learning platforms provide out-of-the-box reproducibility? Future Generation Computer Systems 126
2021
Later among the works it cites.
Muellner, P., Kowald, D., Lex, E.: Robustness of meta matrix factorization against strict privacy constraints. In: Advances in Information Retrieval: 43rd European Conference on IR Research, ECIR 2021, Virtual Event, March 28–April 1, 2021, Proceedings, Part II 43. pp. 107–119. Springer (2021)
2021
Later among the works it cites.
Pineau, J., Vincent-Lamarre, P., Sinha, K., Larivière, V., Beygelzimer, A., d’Alché Buc, F., Fox, E., Larochelle, H.: Improving reproducibility in machine learning research (a report from the NeurIPS 2019 reproducibility program). The Journal of Machine Learning Research 22
2021
Later among the works it cites.
Yildiz, B., Hung, H., Krijthe, J., Liem, C., Loog, M., Migut, G., Oliehoek, F., Panichella, A., Pawełczak, P., Picek, S., de Weerdt, M., van Gemert, J.: ReproducedPapers.org: Openly Teaching and Structuring Machine Learning Reproducibility. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 12636 LNCS
2021
Later among the works it cites.
Ahmed, H., Lofstead, J.: Managing Randomness to Enable Reproducible Machine Learning. In: Proceedings of the 5th International Workshop on Practical Reproducible Evaluation of Computer Systems. pp. 15–20. ACM, Minneapolis MN USA (2022). https://doi.org/10.1145/3526062.3536353, https://dl.acm.org/doi/10.1145/3526062.3536353
2022
Later among the works it cites.
Barnett, E., Onete, D., Salekin, A., Faraone, S.V.: Genomic Machine Learning Meta-regression: Insights on Associations of Study Features with Reported Model Performance (2022). https://doi.org/10.1101/2022.01.10.22268751, https://www.medrxiv.org/content/10.1101/2022.01.10.22268751v2 , pages: 2022.01.10.22268751
2022
Later among the works it cites.
2022
Later among the works it cites.
Gibney, E.: Could machine learning fuel a reproducibility crisis in science? Nature 608
2022
Later among the works it cites.
2022
Later among the works it cites.
Shahriari, M., Ramler, R., Fischer, L.: How Do Deep-Learning Framework Versions Affect the Reproducibility of Neural Network Models? Machine Learning and Knowledge Extraction 4
2022
Later among the works it cites.