Fetching the paper…

Reading the bibliography…

2019

Probabilistic Bias Mitigation in Word Embeddings

Joren, Hailey, Alvarez-Melis, David

Original
PDF
Post on X

Understand

It has been shown that word embeddings derived from large corpora tend to incorporate biases present in their training data.

Various methods for mitigating these biases have been proposed, but recent work has demonstrated that these methods hide but fail to truly remove the biases, which can still be observed in word nearest-neighbor statistics.
In this work we propose a probabilistic view of word embedding bias.
We leverage this framework to present a novel method for mitigating bias which relies on probabilistic observations to yield a more robust bias mitigation algorithm.

Probabilistic Bias Mitigation in Word Embeddings · Around

Built on

Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them
Original
Hila Gonen and Yoav Goldberg · 1903
Earlier work this paper cites.
Distributed representations of words and phrases and their compositionality
Original
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean · 2013
Earlier work this paper cites.
Neural word embedding as implicit matrix factorization
Omer Levy and Yoav Goldberg · 2014
Earlier work this paper cites.
Man is to computer programmer as woman is to homemaker? debiasing word embeddings
Original
Tolga Bolukbasi, Kai-Wei Chang, James Y. Zou, Venkatesh Saligrama, and Adam Kalai · 2016
Earlier work this paper cites.

Similar

Semantics derived automatically from language corpora necessarily contain human biases
Original
Aylin Caliskan Islam, Joanna J. Bryson, and Arvind Narayanan · 2016
Cited alongside, same era.
Algorithmic bias? an empirical study into apparent gender-based discrimination in the display of stem career ads
Catherine E. Tucker Anja Lambrecht · 2016
Cited alongside, same era.
Equality of opportunity in supervised learning
Original
Moritz Hardt, Eric Price, and Nathan Srebro · 2016
Cited alongside, same era.
Word embeddings as metric recovery in semantic spaces
Tatsunori B Hashimoto, David Alvarez-Melis, and Tommi S Jaakkola · 2016
Cited alongside, same era.

Then

How to evaluate word embeddings? on importance of data efficiency and simple supervised tasks
Original
Stanislaw Jastrzkebski, Damian Lesniak, and Wojciech Marian Czarnecki · 2017
Later among the works it cites.
Advances in pre-training distributed word representations
Original
Tomas Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, and Armand Joulin · 2017
Later among the works it cites.
Google autocomplete still makes vile suggestions, 2018
Issie Lapowsky · 2018
Later among the works it cites.
Understanding undesirable word embedding associations
Kawin Ethayarajh, David Kristjanson Duvenaud, and Graeme Hirst · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…