Fetching the paper…
Reading the bibliography…
Pretrained Masked Language Models (MLMs) have revolutionised NLP in recent years.
Roberta: A robustly optimized bert pretraining approach
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019 · 1907
Earlier work this paper cites.
Distributional structure
Zellig S Harris. 1954 · 1954
Earlier work this paper cites.
Dropout: a simple way to prevent neural networks from overfitting
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014 · 1958
Earlier work this paper cites.
Scheduled DropHead: A regularization method for transformer models
Wangchunshu Zhou, Tao Ge, Furu Wei, Ming Zhou, and Ke Xu. 2020 · 1980
Earlier work this paper cites.
WordNet
Christiane Fellbaum. 1998 · 1998
Earlier work this paper cites.
The unified medical language system (umls): integrating biomedical terminology
Olivier Bodenreider. 2004 · 2004
Earlier work this paper cites.
Language-agnostic bert sentence embedding
Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2020 · 2007
Earlier work this paper cites.
Domain-specific language model pretraining for biomedical natural language processing
Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. 2020 · 2007
Earlier work this paper cites.
SemEval-2012 task 6: A pilot on semantic textual similarity
Eneko Agirre, Daniel Cer, Mona Diab, and Aitor Gonzalez-Agirre. 2012 · 2012
Earlier work this paper cites.
Clear: Contrastive learning for sentence representation
Zhuofeng Wu, Sinong Wang, Jiatao Gu, Madian Khabsa, Fei Sun, and Hao Ma. 2020 · 2012
Earlier work this paper cites.
*SEM 2013 shared task: Semantic textual similarity
Eneko Agirre, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, and Weiwei Guo. 2013 · 2013
Earlier work this paper cites.
Distributed representations of words and phrases and their compositionality
Tomás Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013b · 2013
Earlier work this paper cites.
SemEval-2014 task 10: Multilingual semantic textual similarity
Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo, Rada Mihalcea, German Rigau, and Janyce Wiebe. 2014 · 2014
Earlier work this paper cites.
Ncbi disease corpus: a resource for disease name recognition and concept normalization
Rezarta Islamaj Doğan, Robert Leaman, and Zhiyong Lu. 2014 · 2014
Earlier work this paper cites.
Representing multilingual data as linked data: the case of BabelNet 2.0
Maud Ehrmann, Francesco Cecconi, Daniele Vannella, John Philip McCrae, Philipp Cimiano, and Roberto Navigli. 2014 · 2014
Earlier work this paper cites.
A SICK cure for the evaluation of compositional distributional semantic models
Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Bernardi, and Roberto Zamparelli. 2014 · 2014
Earlier work this paper cites.
GloVe: Global vectors for word representation
Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014 · 2014
Earlier work this paper cites.
SemEval-2015 task 2: Semantic textual similarity, English, Spanish and pilot on interpretability
Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo, Iñigo Lopez-Gazpio, Montse Maritxalar, Rada Mihalcea, German Rigau, Larraitz Uria, and Janyce Wiebe. 2015 · 2015
Earlier work this paper cites.
Xavier Bouthillier, Kishore Konda, Pascal Vincent, and Roland Memisevic. 2015 · 2015
Earlier work this paper cites.
A large annotated corpus for learning natural language inference
Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015 · 2015
Earlier work this paper cites.
SimLex-999: Evaluating semantic models with (genuine) similarity estimation
Felix Hill, Roi Reichart, and Anna Korhonen. 2015 · 2015
Earlier work this paper cites.
Skip-thought vectors
Ryan Kiros, Yukun Zhu, Ruslan Salakhutdinov, Richard S. Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015 · 2015
Earlier work this paper cites.
PPDB 2.0: Better paraphrase ranking, fine-grained entailment relations, word embeddings, and style classification
Ellie Pavlick, Pushpendre Rastogi, Juri Ganitkevitch, Benjamin Van Durme, and Chris Callison-Burch. 2015 · 2015
Earlier work this paper cites.
SemEval-2016 task 1: Semantic textual similarity, monolingual and cross-lingual evaluation
Eneko Agirre, Carmen Banea, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Rada Mihalcea, German Rigau, and Janyce Wiebe. 2016 · 2016
Earlier work this paper cites.
A latent variable model approach to PMI-based word embeddings
Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. 2016 · 2016
Earlier work this paper cites.
Learning distributed representations of sentences from unlabelled data
Felix Hill, Kyunghyun Cho, and Anna Korhonen. 2016 · 2016
Earlier work this paper cites.
Biocreative v cdr task corpus: a resource for chemical disease relation extraction
Jiao Li, Yueping Sun, Robin J Johnson, Daniela Sciaky, Chih-Hsuan Wei, Robert Leaman, Allan Peter Davis, Carolyn J Mattingly, Thomas C Wiegers, and Zhiyong Lu. 2016 · 2016
Cited alongside, same era.
Normalising medical concepts in social media texts by learning semantic representation
Nut Limsopatham and Nigel Collier. 2016 · 2016
Cited alongside, same era.
SQuAD: 100,000+ questions for machine comprehension of text
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016 · 2016
Cited alongside, same era.
SemEval-2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation
Daniel Cer, Mona Diab, Eneko Agirre, Iñigo Lopez-Gazpio, and Lucia Specia. 2017 · 2017
Cited alongside, same era.
A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings
Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2018 · 2018
Cited alongside, same era.
Augmix: A simple data processing method to improve robustness and uncertainty
Dan Hendrycks, Norman Mu, Ekin Dogus Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshminarayanan. 2020 · 2020
Later among the works it cites.
Poly-encoders: Architectures and pre-training strategies for fast and accurate multi-sentence scoring
Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux, and Jason Weston. 2020 · 2020
Later among the works it cites.
On the sentence embeddings from pre-trained language models
Bohan Li, Hao Zhou, Junxian He, Mingxuan Wang, Yiming Yang, and Lei Li. 2020 · 2020
Later among the works it cites.
A primer in BERTology: What we know about how BERT works
Anna Rogers, Olga Kovaleva, and Anna Rumshisky. 2020 · 2020
Later among the works it cites.
Simbert: Integrating retrieval and generation into bert
Jianlin Su. 2020 · 2020
Later among the works it cites.
Probing pretrained language models for lexical semantics
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Universal sentence encoder for English
Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, Brian Strope, and Ray Kurzweil. 2018 · 2018
Cited alongside, same era.
Word translation without parallel data
Guillaume Lample, Alexis Conneau, Marc’Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. 2018 · 2018
Cited alongside, same era.
An efficient framework for learning sentence representations
Lajanugen Logeswaran and Honglak Lee. 2018 · 2018
Cited alongside, same era.
Advances in pre-training distributed word representations
Tomas Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, and Armand Joulin. 2018 · 2018
Cited alongside, same era.
All-but-the-top: Simple and effective postprocessing for word representations
Jiaqi Mu and Pramod Viswanath. 2018 · 2018
Cited alongside, same era.
Representation learning with contrastive predictive coding
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018 · 2018
Cited alongside, same era.
Card-660: Cambridge rare word dataset - a reliable benchmark for infrequent word representation models
Mohammad Taher Pilehvar, Dimitri Kartsaklis, Victor Prokhorov, and Nigel Collier. 2018 · 2018
Cited alongside, same era.
Ivan Vulić, Edoardo Maria Ponti, Robert Litschko, Goran Glavaš, and Anna Korhonen. 2020b · 2020
Later among the works it cites.
Understanding contrastive representation learning through alignment and uniformity on the hypersphere
Tongzhou Wang and Phillip Isola. 2020 · 2020
Later among the works it cites.
An unsupervised sentence embedding method by mutual information maximization
Yan Zhang, Ruidan He, Zuozhu Liu, Kwan Hui Lim, and Lidong Bing. 2020 · 2020
Later among the works it cites.
Isotropy in the contextual embedding space: Clusters and manifolds
Xingyu Cai, Jiaji Huang, Yuchen Bian, and Kenneth Church. 2021 · 2021
Closest in time.
Semantic re-tuning with contrastive tension
Fredrik Carlsson, Amaru Cuba Gyllensten, Evangelia Gogoulou, Erik Ylipää Hellqvist, and Magnus Sahlgren. 2021 · 2021
Closest in time.
SimCSE: Simple contrastive learning of sentence embeddings
Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021 · 2021
Closest in time.
DeCLUTR: Deep contrastive learning for unsupervised textual representations
John Giorgi, Osvald Nitski, Bo Wang, and Gary Bader. 2021 · 2021
Closest in time.
Is supervised syntactic parsing beneficial for language understanding tasks? an empirical investigation
Goran Glavaš and Ivan Vulić. 2021 · 2021
Closest in time.
Self-guided contrastive learning for BERT sentence representations
Taeuk Kim, Kang Min Yoo, and Sang-goo Lee. 2021 · 2021
Closest in time.
Evaluating multilingual text encoders for unsupervised cross-lingual retrieval
Robert Litschko, Ivan Vulić, Simone Paolo Ponzetto, and Goran Glavaš. 2021 · 2021
Closest in time.
Self-alignment pretraining for biomedical entity representations
Fangyu Liu, Ehsan Shareghi, Zaiqiao Meng, Marco Basaldella, and Nigel Collier. 2021 · 2021
Closest in time.
Coco-lm: Correcting and contrasting text sequences for language model pretraining
Yu Meng, Chenyan Xiong, Payal Bajaj, Saurabh Tiwary, Paul Bennett, Jiawei Han, and Xia Song. 2021 · 2021
Closest in time.
A cluster-based approach for improving isotropy in contextual embedding space
Sara Rajaee and Mohammad Taher Pilehvar. 2021 · 2021
Closest in time.
Nils Rethmeier and Isabelle Augenstein. 2021 · 2021
Closest in time.
WikiMatrix: Mining 135M parallel sentences in 1620 language pairs from Wikipedia
Holger Schwenk, Vishrav Chaudhary, Shuo Sun, Hongyu Gong, and Francisco Guzmán. 2021 · 2021
Closest in time.
Whitening sentence representations for better semantics and faster retrieval
Jianlin Su, Jiarun Cao, Weijie Liu, and Yangyiwen Ou. 2021 · 2021
Closest in time.
Augmented SBERT: Data augmentation method for improving bi-encoders for pairwise sentence scoring tasks
Nandan Thakur, Nils Reimers, Johannes Daxenberger, and Iryna Gurevych. 2021 · 2021
Closest in time.
LexFit: Lexical fine-tuning of pretrained language models
Ivan Vulić, Edoardo Maria Ponti, Anna Korhonen, and Goran Glavaš. 2021 · 2021
Closest in time.
ConSERT: A contrastive framework for self-supervised sentence representation transfer
Yuanmeng Yan, Rumei Li, Sirui Wang, Fuzheng Zhang, Wei Wu, and Weiran Xu. 2021 · 2021
Closest in time.
Bootstrapped unsupervised sentence representation learning
Yan Zhang, Ruidan He, Zuozhu Liu, Lidong Bing, and Haizhou Li. 2021 · 2021
Closest in time.