Inter-Annotator Agreement in the Wild: Uncovering Its Emerging Roles and Considerations in Real-World Scenarios

Data cleaning: Problems and current approaches

Rahm, E., Do, H. H., et al · 2000

Earlier work this paper cites.

Evaluating the impact of pre-annotation on annotation speed and potential bias: natural language processing gold standard development for clinical named entity recognition in clinical trial announcements

Lingren, T., Deleger, L., Molnar, K., Zhai, H., Meinzen-Derr, J., Kaiser, M., Stoutenborough, L., Li, Q., and Solti, I · 2014

Earlier work this paper cites.

Assessing inter-annotator agreement for translation error annotation

Lommel, A., Popovic, M., and Burchardt, A · 2014

Cited alongside, same era.

Inter-annotator agreement

Artstein, R · 2017

Cited alongside, same era.

Identifying annotator bias: A new irt-based method for bias identification

Amidei, J., Piwek, P., and Willis, A · 2020

Later among the works it cites.

Dmops: Data management operation and recipes

Original

Choi, E. and Park, C · 2023

Closest in time.