Fetching the paper…
Reading the bibliography…
Language model pre-training and derived methods are incredibly impactful in machine learning.
ERNIE: Enhanced Representation through Knowledge Integration
Sun, Y. et al · 1904
Earlier work this paper cites.
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y. et al · 1907
Earlier work this paper cites.
Supervised graph inference
Vert, J.-P. & Yamanishi, Y · 2004
Earlier work this paper cites.
Dimensionality Reduction by Learning an Invariant Mapping
Hadsell, R., Chopra, S. & LeCun, Y · 2006
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
Deng, J. et al · 2009
Earlier work this paper cites.
Structure preserving embedding
Shaw, B. & Jebara, T · 2009
Earlier work this paper cites.
Learning a Distance Metric from a Network
Shaw, B., Huang, B. & Jebara, T · 2011
Earlier work this paper cites.
Scikit-learn: Machine learning in Python
Pedregosa, F. et al · 2011
Earlier work this paper cites.
Multiclass learning with simplex coding
Mroueh, Y., Poggio, T., Rosasco, L. & Slotine, J.-J. E · 2012
Earlier work this paper cites.
Knowledge graph embedding by translating on hyperplanes
Wang, Z., Zhang, J., Feng, J. & Chen, Z · 2014
Earlier work this paper cites.
Homophily, structure, and content augmented network representation learning
Zhang, D., Yin, J., Zhu, X. & Zhang, C · 2016
Earlier work this paper cites.
Local fitness landscape of the green fluorescent protein
Sarkisyan et al · 2016
Earlier work this paper cites.
Semi-supervised classification with graph convolutional networks
Kipf, T. N. & Welling, M · 2017
Earlier work this paper cites.
Inductive representation learning on large graphs
Hamilton, W. L., Ying, R. & Leskovec, J · 2017
Earlier work this paper cites.
Global analysis of protein folding using massively parallel design, synthesis, and testing
Rocklin, G. J. et al · 2017
Earlier work this paper cites.
Deep contextualized word representations
Peters, M. E. et al · 2018
Earlier work this paper cites.
Improving language understanding by generative pre-training (2018)
Radford, A., Narasimhan, K., Salimans, T. & Sutskever, I · 2018
Earlier work this paper cites.
Deep attributed network embedding
Gao, H. & Huang, H · 2018
Earlier work this paper cites.
Community detection in attributed graphs: An embedding approach
Li, Y., Sha, C., Huang, X. & Zhang, Y · 2018
Earlier work this paper cites.
DeepSF: deep convolutional neural network for mapping protein sequences to folds
Hou, J., Adhikari, B. & Cheng, J · 2018
Earlier work this paper cites.
Measuring the Evolution of a Scientific Field through Citation Frames
Jurgens, D., Kumar, S., Hoover, R., McFarland, D. & Jurafsky, D · 2018
Earlier work this paper cites.
Graph attention networks
Veličković, P. et al · 2018
Earlier work this paper cites.
Rotate: Knowledge graph embedding by relational rotation in complex space
Sun, Z., Deng, Z.-H., Nie, J.-Y. & Tang, J · 2018
Earlier work this paper cites.
Language models are unsupervised multitask learners
Radford, A. et al · 2019
Earlier work this paper cites.
ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators
Clark, K., Luong, M.-T., Le, Q. V. & Manning, C. D · 2019
Earlier work this paper cites.
Unified language model pre-training for natural language understanding and generation
Dong, L. et al · 2019
Earlier work this paper cites.
Knowledge enhanced contextual word representations
Peters, M. E. et al · 2019
Earlier work this paper cites.
Evaluating Protein Transfer Learning with TAPE
Rao, R. et al · 2019
Earlier work this paper cites.
Unified rational protein engineering with sequence-based deep representation learning
Alley, E. C., Khimulya, G., Biswas, S., AlQuraishi, M. & Church, G. M · 2019
Earlier work this paper cites.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K · 2019
Earlier work this paper cites.
ERNIE: Enhanced Language Representation with Informative Entities
Zhang, Z. et al · 2019
Earlier work this paper cites.
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Lan, Z. et al · 2019
Earlier work this paper cites.
Multi-Task Deep Neural Networks for Natural Language Understanding
Liu, X., He, P., Chen, W. & Gao, J · 2019
Earlier work this paper cites.
Evolution of resilience in protein interactomes across the tree of life
Zitnik, M., Sosič, R., Feldman, M. W. & Leskovec, J · 2019
Earlier work this paper cites.
A Review of Microsoft Academic Services for Science of Science Studies
Wang, K. et al · 2019
Earlier work this paper cites.
A Theoretical Analysis of Contrastive Unsupervised Representation Learning
Saunshi, N., Plevrakis, O., Arora, S., Khodak, M. & Khandeparkar, H · 2019
Earlier work this paper cites.
Multi-Similarity Loss With General Pair Weighting for Deep Metric Learning
Wang, X., Han, X., Huang, W., Dong, D. & Scott, M. R · 2019
Earlier work this paper cites.
SciBERT: A Pretrained Language Model for Scientific Text
Beltagy, I., Lo, K. & Cohan, A · 2019
Earlier work this paper cites.
NetSurfP-2.0: Improved prediction of protein structural features by integrated deep learning
Klausen, M. S. et al · 2019
Earlier work this paper cites.
Structural Scaffolds for Citation Intent Classification in Scientific Publications
Cohan, A., Ammar, W., van Zuylen, M. & Cady, F · 2019
Earlier work this paper cites.
Kg-bert: Bert for knowledge graph completion
Yao, L., Mao, C. & Luo, Y · 2019
Earlier work this paper cites.
Sentence-BERT: Sentence embeddings using Siamese BERT-networks
Reimers, N. & Gurevych, I · 2019
Earlier work this paper cites.
Language models are few-shot learners
Brown, T. B. et al · 2020
Earlier work this paper cites.
Exploring the limits of transfer learning with a unified text-to-text transformer
Raffel, C. et al · 2020
Earlier work this paper cites.
BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension
Lewis, M. et al · 2020
Earlier work this paper cites.
Unsupervised cross-lingual representation learning at scale
Conneau, A. et al · 2020
Earlier work this paper cites.
SpanBERT: Improving pre-training by representing and predicting spans
Joshi, M. et al · 2020
Earlier work this paper cites.
Don’t stop pretraining: Adapt language models to domains and tasks
Gururangan, S. et al · 2020
Earlier work this paper cites.
Luke: Deep contextualized entity representations with entity-aware self-attention
Yamada, I., Asai, A., Shindo, H., Takeda, H. & Matsumoto, Y · 2020
Earlier work this paper cites.
Pretrained encyclopedia: Weakly supervised knowledge-pretrained language model
Xiong, W., Du, J., Wang, W. Y. & Stoyanov, V · 2020
Cited alongside, same era.
CoLAKE: Contextualized language and knowledge embedding
Sun, T. et al · 2020
Cited alongside, same era.
BERT-MK: Integrating graph contextualized knowledge into pre-trained language models
He, B. et al · 2020
Cited alongside, same era.
Kgplm: Knowledge-guided language model pre-training via generative and discriminative learning
He, B., Jiang, X., Xiao, J. & Liu, Q · 2020
Cited alongside, same era.
Syntactic structure distillation pretraining for bidirectional encoders
Kuncoro, A. et al · 2020
Cited alongside, same era.
Strategies for Pre-training Graph Neural Networks
DeCLUTR: Deep contrastive learning for unsupervised textual representations
Giorgi, J., Nitski, O., Wang, B. & Bader, G · 2021
Closest in time.
Graph contrastive learning automated
You, Y., Chen, T., Shen, Y. & Wang, Z · 2021
Closest in time.
Coco-lm: Correcting and contrasting text sequences for language model pretraining
Meng, Y. et al · 2021
Closest in time.
Semantic re-tuning with contrastive tension
Carlsson, F., Gyllensten, A. C., Gogoulou, E., Hellqvist, E. Y. & Sahlgren, M · 2021
Closest in time.
InfoXLM: An information-theoretic framework for cross-lingual language model pre-training
Chi, Z. et al · 2021
Closest in time.
KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation
Wang, X. et al · 2021
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Hu, W. et al · 2020
Cited alongside, same era.
SentiLARE: Sentiment-aware language representation learning with linguistic knowledge
Ke, P., Ji, H., Liu, S., Zhu, X. & Huang, M · 2020
Cited alongside, same era.
Pre-Training of Deep Bidirectional Protein Sequence Representations with Structural Information
Min, S., Park, S., Kim, S., Choi, H.-S. & Yoon, S · 2020
Cited alongside, same era.
A Comprehensive Evaluation of Multi-task Learning and Multi-task Pre-training on EHR Time-series Data
McDermott, M. B. A. et al · 2020
Cited alongside, same era.
ERNIE 2.0: A Continual Pre-Training Framework for Language Understanding
Sun, Y. et al · 2020
Cited alongside, same era.
Structbert: Incorporating language structures into pre-training for deep language understanding
Wang, W. et al · 2020
Cited alongside, same era.
Pre-training via paraphrasing
Lewis, M. et al · 2020
Cited alongside, same era.
Knowledge-aware contrastive molecular graph learning
Fang, Y. et al · 2021
Closest in time.
Multitask prompted training enables zero-shot task generalization
Sanh, V. et al · 2021
Closest in time.
Combining analogy with language models for knowledge extraction
Ribeiro, D. N. & Forbus, K · 2021
Closest in time.
Li, M. M., Huang, K. & Zitnik, M · 2021
Closest in time.
SimCSE: Simple contrastive learning of sentence embeddings
Gao, T., Yao, X. & Chen, D · 2021
Closest in time.
A mathematical exploration of why language models help solve downstream tasks
Saunshi, N., Malladi, S. & Arora, S · 2021
Closest in time.
Adversarial contrastive pre-training for protein sequences
McDermott, M., Yap, B., Hsu, H., Jin, D. & Szolovits, P · 2021
Closest in time.
Kg-bart: Knowledge graph-augmented bart for generative commonsense reasoning
Liu, Y., Wan, Y., He, L., Peng, H. & Philip, S. Y · 2021
Closest in time.
Graphformers: Gnn-nested language models for linked text representation
Yang, J. et al · 2021
Closest in time.
Knowledge graph based synthetic corpus generation for knowledge-enhanced language model pre-training
Agarwal, O., Ge, H., Shakeri, S. & Al-Rfou, R · 2021
Closest in time.
Lu, Y., Lu, H., Fu, G. & Liu, Q · 2021
Closest in time.
Drop redundant, shrink irrelevant: Selective knowledge injection for language pretraining
Zhang, N. et al · 2021
Closest in time.
CLEVE: Contrastive Pre-training for Event Extraction
Wang, Z. et al · 2021
Closest in time.
Vilt: Vision-and-language transformer without convolution or region supervision
Kim, W., Son, B. & Kim, I · 2021
Closest in time.
StructuralLM: Structural pre-training for form understanding
Li, C. et al · 2021
Closest in time.
Self-alignment pretraining for biomedical entity representations
Liu, F., Shareghi, E., Meng, Z., Basaldella, M. & Collier, N · 2021
Closest in time.
Pre-Training for Ad-Hoc Retrieval: Hyperlink is Also You Need , 1212–1221 (Association for Computing Machinery, New York, NY, USA, 2021)
Ma, Z. et al · 2021
Closest in time.
Wikipedia entities as rendezvous across languages: Grounding multilingual language models by predicting Wikipedia hyperlinks
Calixto, I., Raganato, A. & Pasini, T · 2021
Closest in time.
Contrastive pre-training of gnns on heterogeneous graphs
Jiang, X., Lu, Y., Fang, Y. & Shi, C · 2021
Closest in time.
Mixture-of-partitions: Infusing large biomedical knowledge graphs into bert
Meng, Z., Liu, F., Clark, T. H., Shareghi, E. & Collier, N · 2021
Closest in time.
ConSERT: A contrastive framework for self-supervised sentence representation transfer
Yan, Y. et al · 2021
Closest in time.
Ki-bert: Infusing knowledge context for better language and domain understanding
Faldu, K., Sheth, A., Kikani, P. & Akabari, H · 2021
Closest in time.
A general method for transferring explicit knowledge into language model pretraining
Yan, R., Sun, L., Wang, F. & Zhang, X · 2021
Closest in time.
Supervised contrastive learning for pre-trained language model fine-tuning
Gunel, B., Du, J., Conneau, A. & Stoyanov, V · 2021
Closest in time.
Self-guided contrastive learning for BERT sentence representations
Kim, T., Yoo, K. M. & Lee, S.-g · 2021
Closest in time.
Whitening sentence representations for better semantics and faster retrieval
Su, J., Cao, J., Liu, W. & Ou, Y · 2021
Closest in time.
WhiteningBERT: An easy unsupervised sentence embedding approach
Huang, J. et al · 2021
Closest in time.
Sakg-bert: Enabling language representation with knowledge graphs for chinese sentiment analysis
Yan, X., Jian, F. & Sun, B · 2021
Closest in time.
Multitask prompted training enables zero-shot task generalization
Sanh, V. et al · 2022
Closest in time.
Jaket: Joint pre-training of knowledge graph and language understanding
Yu, D., Zhu, C., Yang, Y. & Zeng, M · 2022
Closest in time.
The inductive bias of in-context learning: Rethinking pretraining example design
Levine, Y. et al · 2022
Closest in time.
LP-BERT: multi-task pre-training knowledge graph BERT for link prediction
Li, D., Yi, M. & He, Y · 2022
Closest in time.
Dict-BERT: Enhancing language model pre-training with dictionary
Yu, W. et al · 2022
Closest in time.
LinkBERT: Pretraining language models with document links
Yasunaga, M., Leskovec, J. & Liang, P · 2022
Closest in time.
Protein representation learning by geometric structure pretraining
Zhang, Z. et al · 2022
Closest in time.
Molecular contrastive learning with chemical element knowledge graph
Fang, Y. et al · 2022
Closest in time.
Xlm-k: Improving cross-lingual language model pre-training with multilingual knowledge
Jiang, X., Liang, Y., Chen, W. & Duan, N · 2022
Closest in time.
Webformer: Pre-training with web pages for information retrieval
Guo, Y. et al · 2022
Closest in time.
SimKGC: Simple contrastive knowledge graph completion with pre-trained language models
Wang, L., Zhao, W., Wei, Z. & Liu, J · 2022
Closest in time.
A unified continuous learning framework for multi-modal knowledge discovery and pre-training
Fan, Z. et al · 2022
Closest in time.
Code: Contrastive pre-training with adversarial fine-tuning for zero-shot expert linking
Chen, B. et al · 2022
Closest in time.
GreaseLM: Graph REASoning enhanced language models
Zhang, X. et al · 2022
Closest in time.