Fetching the paper…
Reading the bibliography…
Majority voting and averaging are common approaches employed to resolve annotator disagreements and derive single ground truth labels from multiple annotations.
Tackling online abuse: A survey of automated abuse detection methods
Pushkar Mishra, Helen Yannakoudakis, and Ekaterina Shutova. 2019 · 1908
Earlier work this paper cites.
Maximum likelihood estimation of observer error-rates using the em algorithm
Alexander Philip Dawid and Allan M Skene. 1979 · 1979
Earlier work this paper cites.
A general psychoevolutionary theory of emotion
Robert Plutchik. 1980 · 1980
Earlier work this paper cites.
An argument for basic emotions
Paul Ekman. 1992 · 1992
Earlier work this paper cites.
Empathy, ways of knowing, and interdependence as mediators of gender differences in attitudes toward hate speech and freedom of speech
Gloria Cowan and Désirée Khatchadourian. 2003 · 2003
Earlier work this paper cites.
Experiments in emotional speech
Julia Hirschberg, Jackson Liscombe, and Jennifer Venditti. 2003 · 2003
Earlier work this paper cites.
Classifying subject ratings of emotional speech using acoustic features
Jackson Liscombe, Jennifer Venditti, and Julia Hirschberg. 2003 · 2003
Earlier work this paper cites.
A model of textual affect sensing using real-world knowledge
Hugo Liu, Henry Lieberman, and Ted Selker. 2003 · 2003
Earlier work this paper cites.
Core affect and the psychological construction of emotion
James A Russell. 2003 · 2003
Earlier work this paper cites.
A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts
Bo Pang and Lillian Lee. 2004 · 2004
Earlier work this paper cites.
Learning subjective language
Janyce Wiebe, Theresa Wilson, Rebecca Bruce, Matthew Bell, and Melanie Martin. 2004 · 2004
Earlier work this paper cites.
A corpus-based approach to finding happiness
Rada Mihalcea and Hugo Liu. 2006 · 2006
Earlier work this paper cites.
Identifying expressions of emotion in text
Saima Aman and Stan Szpakowicz. 2007 · 2007
Earlier work this paper cites.
Semeval-2007 task 14: Affective text
Carlo Strapparava and Rada Mihalcea. 2007 · 2007
Earlier work this paper cites.
Affect in* Text and Speech
Ebba Cecilia Ovesdotter Alm. 2008 · 2008
Earlier work this paper cites.
Cheap and fast – but is it good? evaluating non-expert annotations for natural language tasks
Rion Snow, Brendan O’Connor, Daniel Jurafsky, and Andrew Ng. 2008 · 2008
Earlier work this paper cites.
Interpreting ambiguous emotional expressions
Emily Mower, Angeliki Metallinou, Chi-Chun Lee, Abe Kazemzadeh, Carlos Busso, Sungbok Lee, and Shrikanth Narayanan. 2009 · 2009
Earlier work this paper cites.
Sentiment analysis and subjectivity
Bing Liu et al. 2010 · 2010
Earlier work this paper cites.
How reliable are annotations via crowdsourcing: a study about inter-annotator agreement for multi-label image annotation
Stefanie Nowak and Stefan Rüger. 2010 · 2010
Earlier work this paper cites.
Who are the crowdworkers? shifting demographics in mechanical turk
Joel Ross, Lilly Irani, M Six Silberman, Andrew Zaldivar, and Bill Tomlinson. 2010 · 2010
Earlier work this paper cites.
Subjective natural language problems: Motivations, applications, characterizations, and implications
Cecilia Ovesdotter Alm. 2011 · 2011
Earlier work this paper cites.
Agreement and information in the reliability of coding
Klaus Krippendorff. 2011 · 2011
Earlier work this paper cites.
Did it happen? the pragmatic complexity of veridicality assessment
Marie-Catherine De Marneffe, Christopher D Manning, and Christopher Potts. 2012 · 2012
Earlier work this paper cites.
Statistical modality tagging from rule-based annotations and crowdsourcing
Vinodkumar Prabhakaran, Michael Bloodgood, Mona Diab, Bonnie Dorr, Lori Levin, Christine D. Piatko, Owen Rambow, and Benjamin Van Durme. 2012 · 2012
Earlier work this paper cites.
Detecting hate speech on the world wide web
William Warner and Julia Hirschberg. 2012 · 2012
Earlier work this paper cites.
Crowd truth: Harnessing disagreement in crowdsourcing a relation extraction gold standard
Lora Aroyo and Chris Welty. 2013 · 2013
Earlier work this paper cites.
Modelling annotator bias with multi-task Gaussian processes: An application to machine translation quality estimation
Trevor Cohn and Lucia Specia. 2013 · 2013
Earlier work this paper cites.
Emotion detection in suicide notes
Bart Desmet and Véronique Hoste. 2013 · 2013
Earlier work this paper cites.
Learning whom to trust with mace
Dirk Hovy, Taylor Berg-Kirkpatrick, Ashish Vaswani, and Eduard Hovy. 2013 · 2013
Earlier work this paper cites.
“sure, i did the right thing”: a system for sarcasm detection in speech
Rachel Rakov and Andrew Rosenberg. 2013 · 2013
Earlier work this paper cites.
Pay by the bit: an information-theoretic metric for collective human judgment
Tamsyn P Waterhouse. 2013 · 2013
Earlier work this paper cites.
The benefits of a model of annotation
Rebecca J Passonneau and Bob Carpenter. 2014 · 2014
Cited alongside, same era.
Learning part-of-speech taggers with inter-annotator agreement loss
Barbara Plank, Dirk Hovy, and Anders Søgaard. 2014 · 2014
Cited alongside, same era.
Corpus annotation through crowdsourcing: Towards best practice guidelines
Marta Sabou, Kalina Bontcheva, Leon Derczynski, and Arno Scharl. 2014 · 2014
Cited alongside, same era.
Predicting word sense annotation agreement
Héctor Martínez Alonso, Anders Johannsen, Oier Lopez de Lacalle, and Eneko Agirre. 2015 · 2015
Cited alongside, same era.
Crowdsourcing disagreement for collecting semantic annotation
Anca Dumitrache. 2015 · 2015
Cited alongside, same era.
Advances in natural language processing
Julia Hirschberg and Christopher D Manning. 2015 · 2015
Cited alongside, same era.
Bert: Pre-training of deep bidirectional transformers for language understanding
J. Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Later among the works it cites.
Pushshift gab corpus
Gavin Gaffney. 2018 · 2019
Later among the works it cites.
Are we modeling the task or the annotator? an investigation of annotator bias in natural language understanding datasets
Mor Geva, Yoav Goldberg, and Jonathan Berant. 2019 · 2019
Later among the works it cites.
Characterizing sources of uncertainty to proxy calibration and disambiguate annotator and data bias
Asma Ghandeharioun, Brian Eoff, Brendan Jou, and Rosalind Picard. 2019 · 2019
Later among the works it cites.
A just and comprehensive strategy for using NLP to address online abuse
David Jurgens, Libby Hemphill, and Eshwar Chandrasekharan. 2019 · 2019
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Adam: A method for stochastic optimization
Diederik P. Kingma and Jimmy Ba. 2015 · 2015
Cited alongside, same era.
https://www.figureeight.com/data/sentiment-analysis-emotion-text/
Crowdflower. 2016 · 2016
Cited alongside, same era.
Modeling subjectiveness in emotion recognition with deep neural networks: Ensembles vs soft labels
Haytham M Fayek, Margaret Lech, and Lawrence Cavedon. 2016 · 2016
Cited alongside, same era.
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Yarin Gal and Zoubin Ghahramani. 2016 · 2016
Cited alongside, same era.
Parting crowds: Characterizing divergent interpretations in crowdsourced annotation tasks
Sanjay Kairam and Jeffrey Heer. 2016 · 2016
Cited alongside, same era.
Are you a racist or am i seeing things? annotator influence on hate speech detection on twitter
Zeerak Waseem. 2016 · 2016
Cited alongside, same era.
Multi-task deep neural networks for natural language understanding
Xiaodong Liu, Pengcheng He, Weizhu Chen, and Jianfeng Gao. 2019 · 2019
Later among the works it cites.
A bert-based transfer learning approach for hate speech detection in online social media
Marzieh Mozafari, Reza Farahbakhsh, and Noel Crespi. 2019 · 2019
Later among the works it cites.
Annotating social media data from vulnerable populations: Evaluating disagreement between domain experts and graduate student annotators
Desmond Patton, Philipp Blandfort, William Frey, Michael Gaskell, and Svebor Karaman. 2019 · 2019
Later among the works it cites.
Emotion recognition in conversation: Research challenges, datasets, and recent advances
Soujanya Poria, Navonil Majumder, Rada Mihalcea, and Eduard Hovy. 2019 · 2019
Later among the works it cites.
Perturbation sensitivity analysis to detect unintended model biases
Vinodkumar Prabhakaran, Ben Hutchinson, and Margaret Mitchell. 2019 · 2019
Later among the works it cites.
The risk of racial bias in hate speech detection
Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A Smith. 2019 · 2019
Later among the works it cites.
CXPlain: Causal Explanations for Model Interpretation under Uncertainty
Patrick Schwab and Walter Karlen. 2019 · 2019
Later among the works it cites.
A multilingual evaluation for online hate speech detection
Michele Corazza, Stefano Menini, Elena Cabrio, Sara Tonelli, and Serena Villata. 2020 · 2020
Later among the works it cites.
GoEmotions: A dataset of fine-grained emotions
Dorottya Demszky, Dana Movshovitz-Attias, Jeongwoo Ko, Alan Cowen, Gaurav Nemade, and Sujith Ravi. 2020 · 2020
Later among the works it cites.
Biases as Values: Evaluating Algorithms in Context
Mark Díaz. 2020 · 2020
Later among the works it cites.
Social biases in NLP models as barriers for persons with disabilities
Ben Hutchinson, Vinodkumar Prabhakaran, Emily Denton, Kellie Webster, Yu Zhong, and Stephen Denuyl. 2020 · 2020
Later among the works it cites.
The gab hate corpus: A collection of 27k posts annotated for hate speech
Brendan Kennedy, Mohammad Atari, Aida Mostafazadeh Davani, Leigh Yeh, Ali Omrani, Yehsong Kim, Kris Coombs Jr., Shreya Havaldar, Gwenyth Portillo-Wightman, Elaine Gonzalez, Joe Hoover, Aida Azatian, Gabriel Cardenas, Alyzeh Hussain, Austin Lara, Adam Omary, Christina Park, Xin Wang, Clarisa Wijaya, Yong Zhang, Beth Meyerowitz, and Morteza Dehghani. 2020 · 2020
Later among the works it cites.
Human-in-the-loop learning from crowdsourcing and social media
Tong Liu. 2020 · 2020
Later among the works it cites.
Detecting stance in media on global warming
Yiwei Luo, Dallas Card, and Dan Jurafsky. 2020 · 2020
Later among the works it cites.
Online abuse and human rights: WOAH satellite session at RightsCon 2020
Vinodkumar Prabhakaran, Zeerak Waseem, Seyi Akiwowo, and Bertie Vidgen. 2020 · 2020
Later among the works it cites.
Six attributes of unhealthy conversations
Ilan Price, Jordan Gifford-Moore, Jory Flemming, Saul Musker, Maayan Roichman, Guillaume Sylvain, Nithum Thain, Lucas Dixon, and Jeffrey Sorensen. 2020 · 2020
Later among the works it cites.
Transformers: State-of-the-art natural language processing
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020 · 2020
Later among the works it cites.
Identifying personal experience tweets of medication effects using pre-trained RoBERTa language model and its updating
Minghao Zhu, Youzhe Song, Ge Jin, and Keyuan Jiang. 2020 · 2020
Later among the works it cites.
Beyond black & white: Leveraging annotator disagreement via soft-label multi-task learning
Tommaso Fornaciari, Alexandra Uma, Silviu Paun, Barbara Plank, Dirk Hovy, and Massimo Poesio. 2021 · 2021
Closest in time.
The disagreement deconvolution: Bringing machine learning performance metrics in line with reality
Mitchell L Gordon, Kaitlyn Zhou, Kayur Patel, Tatsunori Hashimoto, and Michael S Bernstein. 2021 · 2021
Closest in time.
Toxic comment classification challenge
Jigsaw. 2018 · 2021
Closest in time.
Unintended bias in toxicity classification
Jigsaw. 2019 · 2021
Closest in time.
On releasing annotator-level labels and information in datasets
Vinodkumar Prabhakaran, Aida Mostafazadeh Davani, and Mark Diaz. 2021 · 2021
Closest in time.
Learning from the worst: Dynamically generated datasets to improve online hate detection
Bertie Vidgen, Tristan Thrush, Zeerak Waseem, and Douwe Kiela. 2021 · 2021
Closest in time.
Challenges in automated debiasing for toxic language detection
Xuhui Zhou, Maarten Sap, Swabha Swayamdipta, Yejin Choi, and Noah Smith. 2021 · 2021
Closest in time.