Inferring ground truth from multi-annotator ordinal data: a probabilistic approach

Maximum likelihood estimation of observer error-rates using the EM algorithm

A.P. Dawid and A.M. Skene · 1979

Earlier work this paper cites.

Inferring ground truth from subjective labelling of venus images

P. Smyth, U. Fayyad, M. Burl, P. Perona, and P. Baldi · 1995

Earlier work this paper cites.

A simple approach to ordinal classification

E. Frank and M. Hall · 2001

Earlier work this paper cites.

Variational Algorithms for Approximate Bayesian Inference

M.J. Beal · 2003

Earlier work this paper cites.

Gaussian processes for ordinal regression

W. Chu and Z. Ghahramani · 2005

Earlier work this paper cites.

Variational bayesian multinomial probit regression with gaussian process priors

M. Girolami and S. Rogers · 2006

Earlier work this paper cites.

The rise of crowdsourcing

J. Howe · 2006

Earlier work this paper cites.

Multilevel Bayesian models of categorical data annotation

B. Carpenter · 2008

Cited alongside, same era.

Utility data annotation with amazon mechanical turk

A. Sorokin and D. Forsyth · 2008

Cited alongside, same era.

Cheap and fast—but is it good?: Evaluating non-expert annotations for natural language tasks

R. Snow, B. O’Connor, D. Jurafsky, and A.Y. Ng · 2008

Cited alongside, same era.

Learning to rank for information retrieval

T.Y. Liu · 2009

Cited alongside, same era.

Supervised learning from multiple experts: Whom to trust when everyone lies a bit

V.C. Raykar, S. Yu, L.H. Zhao, A. Jerebko, C. Florin, G.H. Valadez, L. Bogoni, and L. Moy · 2009

Cited alongside, same era.

Whose vote should count more: Optimal integration of labels from labelers of unknown expertise

J. Whitehill, P. Ruvolo, T. Wu, J. Bergsma, and J. Movellan · 2009

Semi-parametric analysis of multi-rater data

S. Rogers, M. Girolami, and T. Polajnar · 2010

Later among the works it cites.

Learning from crowds

V.C. Raykar, S. Yu, L.H. Zhao, G.H. Valadez, C. Florin, L. Bogoni, and L. Moy · 2010

Later among the works it cites.

The multidimensional wisdom of crowds

P. Welinder, S. Branson, S. Belongie, and P. Perona · 2010

Later among the works it cites.

Modeling annotator expertise: Learning when everybody knows a bit of something

Y. Yan, R. Rosales, G. Fung, M. Schmidt, G. Hermosillo, L. Bogoni, L. Moy, and J. Dy · 2010

Later among the works it cites.

Iterative learning for reliable crowdsourcing systems

D.R. Karger, S. Oh, and D. Shah · 2011

Later among the works it cites.

Annotation models for crowdsourced ordinal data

V.C. Raykar and S. Yu · 2011

Later among the works it cites.