Error Rate Bounds and Iterative Weighted Majority Voting for Crowdsourcing

On the distribution of the number of successes in independent trials

W. Hoeffding · 1956

Earlier work this paper cites.

Maximum likelihood from incomplete data via the em algorithm

A. P. Dempster, N. M. Laird, and D. B. Rubin · 1977

Earlier work this paper cites.

Maximum Likelihood Estimation of Observer Error-Rates Using the EM Algorithm

A. P. Dawid and A. M. Skene · 1979

Earlier work this paper cites.

Learning from noisy examples

D. Angluin and P. Laird · 1988

Earlier work this paper cites.

Inferring Ground Truth from Subjective Labelling of Venus Images

P. Smyth, U. Fayyad, M. Burl, P. Perona, and P. Baldi · 1995

Earlier work this paper cites.

A Gentle Tutorial of the EM Algorithm and its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models

J. A. Bilmes · 1998

Earlier work this paper cites.

Concentration

C. McDiarmid · 1998

Earlier work this paper cites.

Learning with Multiple Labels

R. Jin and Z. Ghahramani · 2002

Earlier work this paper cites.

Get Another Label? Improving Data Quality and Data Mining Using Multiple, Noisy Labelers Categories and Subject Descriptors

C. S. Sheng and F. Provost · 2008

Earlier work this paper cites.

Cheap and Fast - But is it Good ? Evaluating Non-Expert Annotations for Natural Language Tasks

R. Snow, B. O. Connor, D. Jurafsky, and A. Y. Ng · 2008

Earlier work this paper cites.

Good learners for evil teachers

O. Dekel and O. Shamir · 2009

Cited alongside, same era.

Whose Vote Should Count More : Optimal Integration of Labels from Labelers of Unknown Expertise

J. Whitehill, P. Ruvolo, T. Wu, J. Bergsma, and J. Movellan · 2009

Cited alongside, same era.

Old and new concentration inequalities, Chapter 2 in Complex Graphs and Networks

F. Chung and L. Liu · 2010

Cited alongside, same era.

Learning From Crowds

V. C. Raykar, S. Yu, L. H. Zhao, C. Florin, L. Bogoni, and L. Moy · 2010

Cited alongside, same era.

The Multidimensional Wisdom of Crowds

P. Welinder, S. Branson, S. Belongie, and P. Perona · 2010

Cited alongside, same era.

Modeling annotator expertise : Learning when everybody knows a bit of something

Y. Yan, R. Rosales, G. Fung, M. Schmidt, G. Hermosillo, L. Bogoni, L. Moy, and J. G. Dy · 2010

Cited alongside, same era.

How to grade a test without knowing the answers — a bayesian graphical model for adaptive crowdsourcing and aptitude testing

Y. Bachrach, T. Graepel, T. Minka, and J. Guiver · 2012

Later among the works it cites.

Pattern classification

R. O. Duda, P. E. Hart, and D. G. Stork · 2012

Later among the works it cites.

Variational Inference for Crowdsourcing

Q. Liu, J. Peng, and A. Ihler · 2012

Later among the works it cites.

Learning from the Wisdom of Crowds by Minimax Entropy

Original

D. Zhou, J. Platt, S. Basu, and Y. Mao · 2012

Later among the works it cites.

Optimistic Knowledge Gradient Policy for Optimal Budget Allocation in Crowdsourcing

X. Chen, Q. Lin, and D. Zhou · 2013

Later among the works it cites.

Aggregating crowdsourced binary ratings

N. Dalvi, A. Dasgupta, R. Kumar, and V. Rastogi · 2013