Human and LLM Biases in Hate Speech Annotations: A Socio-Demographic Analysis of Annotators and Targets

Understand

The rise of online platforms exacerbated the spread of hate speech, demanding scalable and effective detection.

• However, the accuracy of hate speech detection systems heavily relies on human-labeled data, which is inherently susceptible to biases.
• While previous work has examined the issue, the interplay between the characteristics of the annotator and those of the target of the hate are still unexplored.
• We fill this gap by leveraging an extensive dataset with rich socio-demographic information of both annotators and targets, uncovering how human biases manifest in relation to the target's attributes.