Quantifying the Impact of Cognitive Biases in Question-Answering Systems

Understand

Crowdsourcing can identify high-quality solutions to problems; however, individual decisions are constrained by cognitive biases.

• We investigate some of these biases in an experimental model of a question-answering system.
• In both natural and controlled experiments, we observe a strong position bias in favor of answers appearing earlier in a list of choices.
• This effect is enhanced by three cognitive factors: the attention an answer receives, its perceived popularity, and cognitive load, measured by the number of choices a user has to process.