Fetching the paper…
Reading the bibliography…
Biomedical image analysis algorithm validation depends on high-quality annotation of reference datasets, for which labeling instructions are key.
Measures of the amount of ecologic association between species
Lee R Dice. 1945 · 1945
Earlier work this paper cites.
Grounding in communication
Herbert H. Clark and Susan E. Brennan. 1991 · 1991
Earlier work this paper cites.
Instructional manipulation checks: Detecting satisficing to increase statistical power
Daniel M Oppenheimer, Tom Meyvis, and Nicolas Davidenko. 2009 · 2009
Earlier work this paper cites.
How reliable are annotations via crowdsourcing: a study about inter-annotator agreement for multi-label image annotation. In Proceedings of the International Conference on Multimedia Information Retrieval . 557–566
Stefanie Nowak and Stefan Rüger. 2010 · 2010
Earlier work this paper cites.
Crowdsourcing Malaria Parasite Quantification: An Online Game for Analyzing Images of Infected Thick Blood Smears
Miguel Angel Luengo-Oroz, Asier Arranz, and John Frean. 2012 · 2012
Earlier work this paper cites.
Distributed Medical Image Analysis and Diagnosis through Crowd-Sourced Games: A Malaria Case Study
Sam Mavandadi, Stoyan Dimitrov, Steve Feng, Frank Yu, Uzair Sikora, Oguzhan Yaglidere, Swati Padmanabhan, Karin Nielsen, and Aydogan Ozcan. 2012 · 2012
Earlier work this paper cites.
Analyzing Crowd Labor and Designing Incentives for Humans in the Loop
O. Tokarchuk, R. Cuel, and M. Zamarian. 2012 · 2012
Earlier work this paper cites.
Crowdsourcing Quality-of-Experience Assessments
T. Hossfeld, C. Keimel, and C. Timmerer. 2014 · 2014
Earlier work this paper cites.
Can masses of non-experts train highly accurate image classifiers?. In International Conference on Medical Image Computing and Computer-assisted Intervention . 438–445
Lena Maier-Hein, Sven Mersmann, Daniel Kondermann, Sebastian Bodenstedt, Alexandro Sanchez, Christian Stock, Hannes Gotz Kenngott, Mathias Eisenmann, and Stefanie Speidel. 2014 · 2014
Earlier work this paper cites.
Crowdsourcing to Assess Surgical Skill
Thomas S. Lendvay, Lee White, and Timothy Kowalewski. 2015 · 2015
Earlier work this paper cites.
The relationship between motivation, monetary compensation, and data quality among US- and India-based workers on Mechanical Turk
Leib Litman, Jonathan Robinson, and Cheskie Rosenzweig. 2015 · 2015
Earlier work this paper cites.
Playsourcing: A Novel Concept for Knowledge Creation in Biomedical Research. In Deep Learning and Data Labeling for Medical Applications , Gustavo Carneiro, Diana Mateus, Loïc Peter, Andrew Bradley, João Manuel R. S. Tavares, Vasileios Belagiannis, João Paulo Papa, Jacinto C. Nascimento, Marco Loog, Zhi Lu, Jaime S. Cardoso, and Julien Cornebise (Eds.). 269–277
Shadi Albarqouni, Stefan Matl, Maximilian Baust, Nassir Navab, and Stefanie Demirci. 2016 · 2016
Earlier work this paper cites.
A two-part mixed-effects model for analyzing longitudinal microbiome compositional data
Eric Z. Chen and Hongzhe Li. 2016 · 2016
Earlier work this paper cites.
Early Experiences with Crowdsourcing Airway Annotations in Chest CT
Veronika Cheplygina, Adria Perez-Rovira, Wieying Kuo, Harm A. W. M. Tiddens, and Marleen de Bruijne. 2016 · 2016
Earlier work this paper cites.
An Empirical Study Into Annotator Agreement, Ground Truth Estimation, and Algorithm Evaluation
Thomas A. Lampert, André Stumpf, and Pierre Gançarski. 2016 · 2016
Earlier work this paper cites.
Revolt: Collaborative Crowdsourcing for Labeling Machine Learning Datasets. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems . 2334–2346
Joseph Chee Chang, Saleema Amershi, and Ece Kamar. 2017 · 2017
Earlier work this paper cites.
Medical image data and datasets in the era of machine learning—whitepaper from the 2016 C-MIMI meeting dataset session
Marc D Kohli, Ronald M Summers, and J Raymond Geis. 2017 · 2017
Earlier work this paper cites.
Sprout: Crowd-Powered Task Design for Crowdsourcing
Jonathan Bragg, Mausam, and Daniel S. Weld. 2018 · 2018
Earlier work this paper cites.
Mapping of Crowdsourcing in Health: Systematic Review
Perrine Créquit, Ghizlène Mansouri, Mehdi Benchoufi, Alexandre Vivot, and Philippe Ravaud. 2018 · 2018
Earlier work this paper cites.
Large-scale medical image annotation with crowd-powered algorithms
Eric Heim, Tobias Roß, Alexander Seitel, Keno März, Bram Stieltjes, Matthias Eisenmann, Johannes Lebert, Jasmin Metzger, Gregor Sommer, and Alexander W. Sauter. 2018 · 2018
Cited alongside, same era.
Why rankings of biomedical image analysis competitions should be interpreted with care
Lena Maier-Hein, Matthias Eisenmann, Annika Reinke, Sinan Onogur, Marko Stankovic, Patrick Scholz, Tal Arbel, Hrvoje Bogunovic, Andrew P. Bradley, Aaron Carass, Carolin Feldmann, Alejandro F. Frangi, Peter M. Full, Bram van Ginneken, Allan Hanbury, Katrin Honauer, Michal Kozubek, Bennett A. Landman, Keno März, Oskar Maier, Klaus Maier-Hein, Bjoern H. Menze, Henning Müller, Peter F. Neher, Wiro Niessen, Nasir Rajpoot, Gregory C. Sharp, Korsuk Sirinukunwattana, Stefanie Speidel, Christian Stock, Danail Stoyanov, Abdel Aziz Taha, Fons van der Sommen, Ching-Wei Wang, Marc-André Weber, Guoyan Zheng, Pierre Jannin, and Annette Kopp-Schneider. 2018 · 2018
Cited alongside, same era.
Wingit: Efficient refinement of unclear task instructions. In Sixth AAAI Conference on Human Computation and Crowdsourcing
VK Chaithanya Manam and Alexander J Quinn. 2018 · 2018
Cited alongside, same era.
How to exploit weaknesses in biomedical challenge design and organization. In International Conference on Medical Image Computing and Computer-Assisted Intervention . 388–395
Annika Reinke, Matthias Eisenmann, Sinan Onogur, Marko Stankovic, Patrick Scholz, Peter M. Full, Hrvoje Bogunovic, Bennett A. Landman, Oskar Maier, and Bjoern Menze. 2018 · 2018
Emily Denton, Mark Díaz, Ian Kivlichan, Vinodkumar Prabhakaran, and Rachel Rosen. 2021 · 2021
Later among the works it cites.
Iterative Quality Control Strategies for Expert Medical Image Labeling
Beverly Freeman, Naama Hammel, Sonia Phene, Abigail Huang, Rebecca Ackermann, Olga Kanzheleva, Miles Hutson, Caitlin Taggart, Quang Duong, and Rory Sayres. 2021 · 2021
Later among the works it cites.
Datasheets for datasets
Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé Iii, and Kate Crawford. 2021 · 2021
Later among the works it cites.
Heidelberg colorectal data set for surgical data science in the sensor operating room
Lena Maier-Hein, Martin Wagner, Tobias Ross, Annika Reinke, Sebastian Bodenstedt, Peter M. Full, Hellena Hempe, Diana Mindroc-Filimon, Patrick Scholz, Thuy Nuong Tran, Pierangela Bruno, Anna Kisilenko, Benjamin Müller, Tornike Davitashvili, Manuela Capek, Minu D. Tizabi, Matthias Eisenmann, Tim J. Adler, Janek Gröhl, Melanie Schellenberg, Silvia Seidlitz, T. Y. Emmy Lai, Bünyamin Pekdemir, Veith Roethlingshoefer, Fabian Both, Sebastian Bittel, Marc Mengler, Lars Mündermann, Martin Apitz, Annette Kopp-Schneider, Stefanie Speidel, Felix Nickel, Pascal Probst, Hannes G. Kenngott, and Beat P. Müller-Stich. 2021 · 2021
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Deep learning is combined with massive-scale citizen science to improve large-scale image classification
Devin P. Sullivan, Casper F. Winsnes, Lovisa Åkesson, Martin Hjelmare, Mikaela Wiking, Rutger Schutten, Linzi Campbell, Hjalti Leifsson, Scott Rhodes, Andie Nordgren, Kevin Smith, Bernard Revaz, Bergur Finnbogason, Attila Szantner, and Emma Lundberg. 2018 · 2018
Cited alongside, same era.
Inter-observer variability of manual contour delineation of structures in CT
Leo Joskowicz, D. Cohen, N. Caplan, and J. Sosna. 2019 · 2019
Cited alongside, same era.
TaskMate: A Mechanism to Improve the Quality of Instructions in Crowdsourcing. In Companion Proceedings of The 2019 World Wide Web Conference . 1121–1130
V. K. Chaithanya Manam, Dwarakanath Jampani, Mariam Zaim, Meng-Han Wu, and Alexander J. Quinn. 2019 · 2019
Cited alongside, same era.
Comparative validation of multi-instance instrument segmentation in endoscopy: Results of the ROBUST-MIS 2019 challenge
Tobias Roß, Annika Reinke, Peter M. Full, Martin Wagner, Hannes Kenngott, Martin Apitz, Hellena Hempe, Diana Mindroc-Filimon, Patrick Scholz, Thuy Nuong Tran, Pierangela Bruno, Pablo Arbeláez, Gui-Bin Bian, Sebastian Bodenstedt, Jon Lindström Bolmgren, Laura Bravo-Sánchez, Hua-Bin Chen, Cristina González, Dong Guo, Pål Halvorsen, Pheng-Ann Heng, Enes Hosgor, Zeng-Guang Hou, Fabian Isensee, Debesh Jha, Tingting Jiang, Yueming Jin, Kadir Kirtac, Sabrina Kletz, Stefan Leger, Zhixuan Li, Klaus H. Maier-Hein, Zhen-Liang Ni, Michael A. Riegler, Klaus Schoeffmann, Ruohua Shi, Stefanie Speidel, Michael Stenzel, Isabell Twick, Gutai Wang, Jiacheng Wang, Liansheng Wang, Lu Wang, Yujie Zhang, Yan-Jie Zhou, Lei Zhu, Manuel Wiesenfarth, Annette Kopp-Schneider, Beat P. Müller-Stich, and Lena Maier-Hein. 2021b · 2019
Cited alongside, same era.
The state of artificial intelligence-based FDA-approved medical devices and algorithms: an online database
Stan Benjamens, Pranavsingh Dhunnoo, and Bertalan Meskó. 2020 · 2020
Cited alongside, same era.
The challenges of deploying artificial intelligence models in a rapidly evolving pandemic
Yipeng Hu, Joseph Jacob, Geoffrey J. M. Parker, David J. Hawkes, John R. Hurst, and Danail Stoyanov. 2020 · 2020
Cited alongside, same era.
The shape of and solutions to the MTurk quality crisis
Ryan Kennedy, Scott Clifford, Tyler Burleigh, Philip D. Waggoner, Ryan Jewell, and Nicholas J. G. Winter. 2020 · 2020
Cited alongside, same era.
BIAS: Transparent reporting of biomedical image analysis challenges
Lena Maier-Hein, Annika Reinke, Michal Kozubek, Anne L Martel, Tal Arbel, Matthias Eisenmann, Allan Hanbury, Pierre Jannin, Henning Müller, Sinan Onogur, and others. 2020 · 2020
Cited alongside, same era.
Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1)
Curtis G Northcutt, Anish Athalye, and Jonas Mueller. 2021 · 2021
Later among the works it cites.
The PRISMA 2020 statement: an updated guideline for reporting systematic reviews
Matthew J Page, Joanne E McKenzie, Patrick M Bossuyt, Isabelle Boutron, Tammy C Hoffmann, Cynthia D Mulrow, Larissa Shamseer, Jennifer M Tetzlaff, Elie A Akl, Sue E Brennan, Roger Chou, Julie Glanville, Jeremy M Grimshaw, Asbjørn Hróbjartsson, Manoj M Lalu, Tianjing Li, Elizabeth W Loder, Evan Mayo-Wilson, Steve McDonald, Luke A McGuinness, Lesley A Stewart, James Thomas, Andrea C Tricco, Vivian A Welch, Penny Whiting, and David Moher. 2021 · 2021
Later among the works it cites.
Data and its (dis)contents: A survey of dataset development and use in machine learning research
Amandalynne Paullada, Inioluwa Deborah Raji, Emily M. Bender, Emily Denton, and Alex Hanna. 2021 · 2021
Later among the works it cites.
Mitigating Dataset Harms Requires Stewardship: Lessons from 1000 Papers
Kenny Peng, Arunesh Mathur, and Arvind Narayanan. 2021 · 2021
Later among the works it cites.
Tobias Roß, Pierangela Bruno, Annika Reinke, Manuel Wiesenfarth, Lisa Koeppel, Peter M. Full, Bünyamin Pekdemir, Patrick Godau, Darya Trofimova, and Fabian Isensee. 2021a · 2021
Later among the works it cites.
What Your Radiologist Might be Missing: Using Machine Learning to Identify Mislabeled Instances of X-ray Images. In Proceedings of the 54th Hawaii International Conference on System Sciences (HICSS)
Tim Rädsch, Sven Eckhardt, Florian Leiser, Konstantin D Pandl, Scott Thiebes, and Ali Sunyaev. 2021 · 2021
Later among the works it cites.
Designing clinically translatable artificial intelligence systems for high-dimensional medical imaging
Rohan Shad, John P. Cunningham, Euan A. Ashley, Curtis P. Langlotz, and William Hiesinger. 2021 · 2021
Later among the works it cites.
Call for Challenges
The Medical Image Computing and Computer Assisted Intervention Society. 2021a · 2021
Later among the works it cites.
MICCAI registered challenges
The Medical Image Computing and Computer Assisted Intervention Society. 2021b · 2021
Later among the works it cites.
Amazon Mechanical Turk
Amazon Mechanical Turk Inc. 2022 · 2022
Closest in time.
The rise and fall (and rise) of datasets
Nature Machine Intelligence. 2022 · 2022
Closest in time.
Surgical data science – from concepts toward clinical translation
Lena Maier-Hein, Matthias Eisenmann, Duygu Sarikaya, Keno März, Toby Collins, Anand Malpani, Johannes Fallert, Hubertus Feussner, Stamatia Giannarou, Pietro Mascagni, Hirenkumar Nakawala, Adrian Park, Carla Pugh, Danail Stoyanov, Swaroop S. Vedula, Kevin Cleary, Gabor Fichtinger, Germain Forestier, Bernard Gibaud, Teodor Grantcharov, Makoto Hashizume, Doreen Heckmann-Nötzel, Hannes G. Kenngott, Ron Kikinis, Lars Mündermann, Nassir Navab, Sinan Onogur, Tobias Roß, Raphael Sznitman, Russell H. Taylor, Minu D. Tizabi, Martin Wagner, Gregory D. Hager, Thomas Neumuth, Nicolas Padoy, Justin Collins, Ines Gockel, Jan Goedeke, Daniel A. Hashimoto, Luc Joyeux, Kyle Lam, Daniel R. Leff, Amin Madani, Hani J. Marcus, Ozanan Meireles, Alexander Seitel, Dogu Teber, Frank Ückert, Beat P. Müller-Stich, Pierre Jannin, and Stefanie Speidel. 2022 · 2022
Closest in time.
Robust Medical Instrument Segmentation (ROBUST-MIS) Challenge 2019 - syn18779624 - Wiki
Tobias Roß and Annika Reinke. 2019 · 2022
Closest in time.
MOOD 2020: A public Benchmark for Out-of-Distribution Detection and Localization on medical Images
David Zimmerer, Peter M. Full, Fabian Isensee, Paul Jäger, Tim Adler, Jens Petersen, Gregor Köhler, Tobias Ross, Annika Reinke, Antanas Kascenas, Bjørn Sand Jensen, Alison Q. O’Neil, Jeremy Tan, Benjamin Hou, James Batten, Huaqi Qiu, Bernhard Kainz, Nina Shvetsova, Irina Fedulova, Dmitry V. Dylov, Baolun Yu, Jianyang Zhai, Jingtao Hu, Runxuan Si, Sihang Zhou, Siqi Wang, Xinyang Li, Xuerun Chen, Yang Zhao, Sergio Naval Marimont, Giacomo Tarroni, Victor Saase, Lena Maier-Hein, and Klaus Maier-Hein. 2022 · 2022
Closest in time.