2022

A New Generation of Perspective API: Efficient Multilingual Character-level Transformers

Lees, Alyssa, Tran, Vinh Q., Tay, Yi et al.

Understand

On the world wide web, toxic content detectors are a crucial line of defense against potentially hateful and offensive messages.

As such, building highly effective classifiers that enable a safer internet is an important research area.
Moreover, the web is a highly multilingual, cross-cultural community that develops its own lingo over time.
As such, it is crucial to develop models that are effective across a diverse range of languages, usages, and styles.

Built on

SemEval-2019 Task 5: Multilingual Detection of Hate Speech Against Immigrants and Women in Twitter. In Proceedings of the 13th International Workshop on Semantic Evaluation . Association for Computational Linguistics, Minneapolis, Minnesota, USA, 54–63
Valerio Basile, Cristina Bosco, Elisabetta Fersini, Debora Nozza, Viviana Patti, Francisco Manuel Rangel Pardo, Paolo Rosso, and Manuela Sanguinetti. 2019 · 2007
Earlier work this paper cites.
SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations . Association for Computational Linguistics, Brussels, Belgium, 66–71
Taku Kudo and John Richardson. 2018 · 2012
Earlier work this paper cites.
Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter. In Proceedings of the NAACL Student Research Workshop . Association for Computational Linguistics, San Diego, California, 88–93
Zeerak Waseem and Dirk Hovy. 2016 · 2013
Earlier work this paper cites.
Abusive Language Detection in Online User Content. In Proceedings of the 25th International Conference on World Wide Web (WWW ’16) . International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 145–153
Chikashi Nobata, Joel Tetreault, Achint Thomas, Yashar Mehdad, and Yi Chang. 2016 · 2016
Earlier work this paper cites.
Automated Hate Speech Detection and the Problem of Offensive Language. In Proceedings of the 11th International AAAI Conference on Web and Social Media (ICWSM ’17) . AAAI, Palo Alto, CA, 512–515
Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017 · 2017
Earlier work this paper cites.
Attention is All you Need. In Advances in Neural Information Processing Systems , I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates, Inc
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Earlier work this paper cites.
Ex Machina: Personal Attacks Seen at Scale. In Proceedings of the 26th International Conference on World Wide Web (WWW ’17) . International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 1391–1399
Ellery Wulczyn, Nithum Thain, and Lucas Dixon. 2017 · 2017
Earlier work this paper cites.
Towards accurate detection of offensive language in online communication in arabic
Azalden Alakrot, Liam Murray, and Nikola S Nikolov. 2018 · 2018
Earlier work this paper cites.
Measuring and Mitigating Unintended Bias in Text Classification. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society (AIES ’18) . Association for Computing Machinery, New York, NY, USA, 67–73
Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2018 · 2018
Earlier work this paper cites.
All You Need is "Love": Evading Hate Speech Detection. In Proceedings of the 11th ACM Workshop on Artificial Intelligence and Security (AISec ’18) . Association for Computing Machinery, New York, NY, USA, 2–12
Tommi Gröndahl, Luca Pajola, Mika Juuti, Mauro Conti, and N. Asokan. 2018 · 2018
Earlier work this paper cites.
The Gab Hate Corpus: A collection of 27k posts annotated for hate speech
Brendan Kennedy, Mohammad Atari, Aida M Davani, Leigh Yeh, Ali Omrani, Yehsong Kim, Kris Coombs, Shreya Havaldar, Gwenyth Portillo-Wightman, Elaine Gonzalez, and et al. 2018 · 2018
Earlier work this paper cites.
Mesh-TensorFlow: Deep Learning for Supercomputers. In Proceedings of the 32nd International Conference on Neural Information Processing Systems (NIPS’18) . Curran Associates Inc., Red Hook, NY, USA, 10435–10444
Noam Shazeer, Youlong Cheng, Niki Parmar, Dustin Tran, Ashish Vaswani, Penporn Koanantakool, Peter Hawkins, HyoukJoong Lee, Mingsheng Hong, Cliff Young, Ryan Sepassi, and Blake Hechtman. 2018 · 2018
Earlier work this paper cites.

Similar

Adafactor: Adaptive Learning Rates with Sublinear Memory Cost. In Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research) , Jennifer Dy and Andreas Krause (Eds.), Vol. 80. PMLR, Cambridge, Massachusetts, 4596–4604
Noam Shazeer and Mitchell Stern. 2018 · 2018
Cited alongside, same era.
Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification. In Companion Proceedings of The 2019 World Wide Web Conference (WWW ’19) . Association for Computing Machinery, New York, NY, USA, 491–500
Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2019 · 2019
Cited alongside, same era.
Machine learning for email spam filtering: review, approaches and open research problems
Emmanuel Gbenga Dada, Joseph Stephen Bassi, Haruna Chiroma, Shafi’i Muhammad Abdulhamid, Adebayo Olusola Adetunmbi, and Opeyemi Emmanuel Ajibuwa. 2019 · 2019
Cited alongside, same era.
Racial Bias in Hate Speech and Abusive Language Detection Datasets. In Proceedings of the Third Workshop on Abusive Language Online . Association for Computational Linguistics, Florence, Italy, 25–35
Thomas Davidson, Debasmita Bhattacharya, and Ingmar Weber. 2019 · 2019
Cited alongside, same era.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) . Association for Computational Linguistics, Minneapolis, Minnesota, 4171–4186
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Cited alongside, same era.
Towards robust toxic content classification
Keita Kurita, Anna Belova, and Antonios Anastasopoulos. 2019 · 2019
Cited alongside, same era.
Overview of the HASOC Track at FIRE 2019: Hate Speech and Offensive Content Identification in Indo-European Languages. In Proceedings of the 11th Forum for Information Retrieval Evaluation (FIRE ’19) . Association for Computing Machinery, New York, NY, USA, 14–17
Thomas Mandl, Sandip Modha, Prasenjit Majumder, Daksh Patel, Mohana Dave, Chintak Mandlia, and Aditya Patel. 2019 · 2019
Cited alongside, same era.
BRUMS at HASOC 2019: Deep Learning Models for Multilingual Hate Speech and Offensive Language Identification.. In FIRE (Working Notes) . CEUR, Aachen, Germany, 199–207
Tharindu Ranasinghe, Marcos Zampieri, and Hansi Hettiarachchi. 2019 · 2019
Cited alongside, same era.
The Risk of Racial Bias in Hate Speech Detection. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics . Association for Computational Linguistics, Florence, Italy, 1668–1678
Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A. Smith. 2019 · 2019
Cited alongside, same era.
TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification. In Findings of the Association for Computational Linguistics: EMNLP 2020 . Association for Computational Linguistics, Online, 1644–1650
Francesco Barbieri, Jose Camacho-Collados, Luis Espinosa Anke, and Leonardo Neves. 2020 · 2020
Cited alongside, same era.
The Meaning and Measurement of Bias: Lessons from Natural Language Processing. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (FAT* ’20) . Association for Computing Machinery, New York, NY, USA, 706
Abigail Z. Jacobs, Su Lin Blodgett, Solon Barocas, Hal Daumé, and Hanna Wallach. 2020 · 2020
Cited alongside, same era.
BERTweet: A pre-trained language model for English Tweets. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations . Association for Computational Linguistics, Online, 9–14
Dat Quoc Nguyen, Thanh Vu, and Anh Tuan Nguyen. 2020 · 2020
Cited alongside, same era.

Then

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020 · 2020
Later among the works it cites.
10 New Languages for Perspective API
Tin Acosta, Alyssa Lees, Daniel Borkan, Jeffrey Sorensen, Alyssa Chvasta, Roelle Thorpe, and Lucy Vasserman. 2021 · 2021
Later among the works it cites.
Perspective API
Google Jigsaw. 2017 · 2021
Later among the works it cites.
Hatemoji: A test suite and adversarially-generated dataset for benchmarking and detecting emoji-based hate. (2021)
H Kirk, B Vidgen, P Röttger, and SA Hale. 2021 · 2021
Later among the works it cites.
WILDS: A Benchmark of in-the-Wild Distribution Shifts. In Proceedings of the 38th International Conference on Machine Learning (Proceedings of Machine Learning Research) , Marina Meila and Tong Zhang (Eds.), Vol. 139. PMLR, 5637–5664
Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, Tony Lee, Etienne David, Ian Stavness, Wei Guo, Berton Earnshaw, Imran Haque, Sara M Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang. 2021 · 2021
Later among the works it cites.
Capturing Covertly Toxic Speech via Crowdsourcing. In Proceedings of the First Workshop on Bridging Human–Computer Interaction and Natural Language Processing . Association for Computational Linguistics, Online, 14–20
Alyssa Lees, Daniel Borkan, Ian Kivlichan, Jorge Nario, and Tesh Goyal. 2021 · 2021
Later among the works it cites.
A Study of Multilingual Toxic Text Detection Approaches under Imbalanced Sample Distribution
Guizhe Song, Degen Huang, and Zhifeng Xiao. 2021 · 2021
Later among the works it cites.
Charformer: Fast character transformers via gradient-based subword tokenization
Yi Tay, Vinh Q Tran, Sebastian Ruder, Jai Gupta, Hyung Won Chung, Dara Bahri, Zhen Qin, Simon Baumgartner, Cong Yu, and Donald Metzler. 2021 · 2021
Later among the works it cites.
Practical Transformer-based Multilingual Text Classification. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Papers . Association for Computational Linguistics, Online, 121–129
Cindy Wang and Michele Banko. 2021 · 2021
Later among the works it cites.
ByT5: Towards a token-free future with pre-trained byte-to-byte models
Original
Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, and Colin Raffel. 2021a · 2021
Later among the works it cites.
mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies . Association for Computational Linguistics, Online, 483–498
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021b · 2021
Later among the works it cites.
Towards generalisable hate speech detection: a review on obstacles and solutions
W Yin and A. Zubiaga. 2021 · 2021
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…