Learning to Remove: Towards Isotropic Pre-trained BERT Embedding

Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)

Original

2013

Earlier work this paper cites.

Hasan, S., Curry, E.: Word re-embedding via manifold dimensionality retention. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP. pp. 321–326 (2017)

2017

Earlier work this paper cites.

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. In: Advances in neural information processing systems, NIPS. pp. 5998–6008 (2017)

2017

Earlier work this paper cites.

Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

Original

2018

Earlier work this paper cites.

Gao, J., He, D., Tan, X., Qin, T., Wang, L., Liu, T.: Representation degeneration problem in training natural language generation models. In: International Conference on Learning Representations, ICLR (2018)

2018

Earlier work this paper cites.

Gong, C., He, D., Tan, X., Qin, T., Wang, L., Liu, T.Y.: Frage: Frequency-agnostic word representation. In: Advances in neural information processing systems, NIPS. pp. 1334–1345 (2018)

2018

Earlier work this paper cites.

Mu, J., Viswanath, P.: All-but-the-top: Simple and effective post-processing for word representations. In: International Conference on Learning Representations, ICLR (2018)

2018

Cited alongside, same era.

Ethayarajh, K.: How contextual are contextualized word representations? comparing the geometry of bert, elmo, and gpt-2 embeddings. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, EMNLP. pp. 55–65 (2019)

2019

Cited alongside, same era.

Karve, S., Ungar, L., Sedoc, J.: Conceptor debiasing of word representations evaluated on weat. In: Proceedings of the First Workshop on Gender Bias in Natural Language Processing. pp. 40–48 (2019)

2019

Cited alongside, same era.

Liu, T., Ungar, L., Sedoc, J.: Unsupervised post-processing of word vectors via conceptor negation. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 33, pp. 6778–6785 (2019)

2019

Cited alongside, same era.

Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al.: Huggingface’s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771 (2019)

Original

2019

Later among the works it cites.

Zhou, T., Sedoc, J., Rodu, J.: Getting in shape: word embedding subspaces. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence, AAAI. pp. 5478–5484. AAAI Press (2019)

2019

Later among the works it cites.

Rogers, A., Kovaleva, O., Rumshisky, A.: A primer in bertology: What we know about how bert works. arXiv preprint arXiv:2002.12327 (2020)

Original

2020

Later among the works it cites.

Wang, L., Huang, J., Huang, K., Hu, Z., Wang, G., Gu, Q.: Improving neural language generation with spectrum control. In: International Conference on Learning Representations, ICLR (2020)

2020

Later among the works it cites.