Fetching the paper…
Reading the bibliography…
Producing the embedding of a sentence in an unsupervised way is valuable to natural language matching and retrieval problems in practice.
Roberta: A robustly optimized bert pretraining approach
Y. Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, M. Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019 · 1907
Earlier work this paper cites.
Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019 · 1910
Earlier work this paper cites.
Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers
Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and M. Zhou. 2020 · 2002
Earlier work this paper cites.
Specter: Document-level representation learning using citation-informed transformers
Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, and Daniel S. Weld. 2020 · 2004
Earlier work this paper cites.
Mpnet: Masked and permuted pre-training for language understanding
K. Song, Xu Tan, Tao Qin, Jianfeng Lu, and T. Liu. 2020 · 2004
Earlier work this paper cites.
Language models are few-shot learners
T. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, G. Krüger, T. Henighan, R. Child, A. Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, E. Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, J. Clark, Christopher Berner, Sam McCandlish, A. Radford, Ilya Sutskever, and Dario Amodei. 2020 · 2005
Earlier work this paper cites.
Deberta: Decoding-enhanced bert with disentangled attention
Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020 · 2006
Earlier work this paper cites.
Squeezebert: What can computer vision teach nlp about efficient neural networks?
Forrest N. Iandola, Albert Eaton Shaw, R. Krishna, and K. Keutzer. 2020 · 2006
Earlier work this paper cites.
Language-agnostic bert sentence embedding
Fangxiaoyu Feng, Yin-Fei Yang, Daniel Matthew Cer, N. Arivazhagan, and Wei Wang. 2020 · 2007
Earlier work this paper cites.
Nandan Thakur, N. Reimers, Johannes Daxenberger, and Iryna Gurevych. 2020 · 2010
Earlier work this paper cites.
Semeval-2012 task 6: A pilot on semantic textual similarity
Eneko Agirre, Daniel Matthew Cer, Mona T. Diab, and A. Gonzalez-Agirre. 2012 · 2012
Earlier work this paper cites.
Negative evidences and co-occurences in image retrieval: The benefit of pca and whitening
H. Jégou and O. Chum. 2012 · 2012
Earlier work this paper cites.
*sem 2013 shared task: Semantic textual similarity
Eneko Agirre, Daniel Matthew Cer, Mona T. Diab, A. Gonzalez-Agirre, and Weiwei Guo. 2013 · 2013
Earlier work this paper cites.
Semeval-2014 task 10: Multilingual semantic textual similarity
Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Matthew Cer, Mona T. Diab, A. Gonzalez-Agirre, Weiwei Guo, R. Mihalcea, German Rigau, and J. Wiebe. 2014 · 2014
Earlier work this paper cites.
A sick cure for the evaluation of compositional distributional semantic models
M. Marelli, S. Menini, M. Baroni, L. Bentivogli, R. Bernardi, and Roberto Zamparelli. 2014 · 2014
Cited alongside, same era.
Glove: Global vectors for word representation
Jeffrey Pennington, R. Socher, and Christopher D. Manning. 2014 · 2014
Cited alongside, same era.
Semeval-2015 task 2: Semantic textual similarity, english, spanish and pilot on interpretability
Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Matthew Cer, Mona T. Diab, A. Gonzalez-Agirre, Weiwei Guo, I. Lopez-Gazpio, M. Maritxalar, R. Mihalcea, German Rigau, L. Uria, and J. Wiebe. 2015 · 2015
Cited alongside, same era.
Semeval-2016 task 1: Semantic textual similarity, monolingual and cross-lingual evaluation
Eneko Agirre, Carmen Banea, Daniel Matthew Cer, Mona T. Diab, A. Gonzalez-Agirre, R. Mihalcea, German Rigau, and J. Wiebe. 2016 · 2016
Cited alongside, same era.
A simple but tough-to-beat baseline for sentence embeddings
Sanjeev Arora, Yingyu Liang, and Tengyu Ma. 2017 · 2017
Cited alongside, same era.
Pytorch: An imperative style, high-performance deep learning library
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019 · 2019
Later among the works it cites.
Sentence-bert: Sentence embeddings using siamese bert-networks
Nils Reimers and Iryna Gurevych. 2019 · 2019
Later among the works it cites.
Bert rediscovers the classical nlp pipeline
Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019 · 2019
Later among the works it cites.
GLUE: A multi-task benchmark and analysis platform for natural language understanding
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019 · 2019
Later among the works it cites.
Parameter-free sentence embedding via orthogonal basis
Ziyi Yang, Chenguang Zhu, and Weizhu Chen. 2019 · 2019
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Semeval-2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation
Daniel Matthew Cer, Mona T. Diab, Eneko Agirre, I. Lopez-Gazpio, and Lucia Specia. 2017 · 2017
Cited alongside, same era.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Cited alongside, same era.
Generalizing and improving bilingual word embedding mappings with a multi-step framework of linear transformations
M. Artetxe, Gorka Labaka, and Eneko Agirre. 2018 · 2018
Cited alongside, same era.
Unsupervised random walk sentence embeddings: A strong but simple baseline
Kawin Ethayarajh. 2018 · 2018
Cited alongside, same era.
All-but-the-top: Simple and effective postprocessing for word representations
Jiaqi Mu and Pramod Viswanath. 2018 · 2018
Cited alongside, same era.
Bert: Pre-training of deep bidirectional transformers for language understanding
J. Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Cited alongside, same era.
What does bert learn about the structure of language?
Ganesh Jawahar, Benoît Sagot, and Djamé Seddah. 2019 · 2019
Cited alongside, same era.
Dialogue response ranking training with large-scale human feedback data
Xiang Gao, Yizhe Zhang, Michel Galley, Chris Brockett, and B. Dolan. 2020 · 2020
Later among the works it cites.
Albert: A lite bert for self-supervised learning of language representations
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020 · 2020
Later among the works it cites.
On the sentence embeddings from pre-trained language models
Bohan Li, Hao Zhou, Junxian He, Mingxuan Wang, Yiming Yang, and Lei Li. 2020 · 2020
Later among the works it cites.
Exploring the limits of transfer learning with a unified text-to-text transformer
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, W. Li, and Peter J. Liu. 2020 · 2020
Later among the works it cites.
Sbert-wk: A sentence embedding method by dissecting bert-based word models
Bin Wang and C.-C. Jay Kuo. 2020 · 2020
Later among the works it cites.
Layoutlm: Pre-training of text and layout for document image understanding
Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, and M. Zhou. 2020 · 2020
Later among the works it cites.
An unsupervised sentence embedding method bymutual information maximization
Yan Zhang, Ruidan He, Zuozhu Liu, Kwan Hui Lim, and Lidong Bing. 2020 · 2020
Later among the works it cites.
Whitening sentence representations for better semantics and faster retrieval
Jianlin Su, Jiarun Cao, Weijie Liu, and Yangyiwen Ou. 2021 · 2021
Closest in time.