Anchoring and Agreement in Syntactic Annotations

Understand

We present a study on two key characteristics of human syntactic annotations: anchoring and agreement.

• Anchoring is a well known cognitive bias in human decision making, where judgments are drawn towards pre-existing values.
• We study the influence of anchoring on a standard approach to creation of syntactic resources where syntactic annotations are obtained via human editing of tagger and parser output.
• Our experiments demonstrate a clear anchoring effect and reveal unwanted consequences, including overestimation of parsing performance and lower quality of annotations in comparison with human-based annotations.