Machine Truth Serum

Understand

Wisdom of the crowd revealed a striking fact that the majority answer from a crowd is often more accurate than any individual expert.

• We observed the same story in machine learning--ensemble methods leverage this idea to combine multiple learning algorithms to obtain better classification performance.
• Among many popular examples is the celebrated Random Forest, which applies the majority voting rule in aggregating different decision trees to make the final prediction.
• Nonetheless, these aggregation rules would fail when the majority is more likely to be wrong.