Fetching the paper…
Reading the bibliography…
Model pre-training on large text corpora has been demonstrated effective for various downstream applications in the NLP domain.
The probable error of a mean
Student. 1908 · 1908
Earlier work this paper cites.
Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics
Xavier Glorot and Yoshua Bengio. 2010 · 2010
Earlier work this paper cites.
Inductive representation learning on large graphs. In Advances in neural information processing systems
Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017 · 2017
Earlier work this paper cites.
Semi-Supervised Classification with Graph Convolutional Networks. In International Conference on Learning Representations
Thomas N Kipf and Max Welling. 2017 · 2017
Earlier work this paper cites.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018 · 2018
Earlier work this paper cites.
Improving language understanding by generative pre-training
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al · 2018
Earlier work this paper cites.
Modeling relational data with graph convolutional networks. In The Semantic Web: 15th International Conference, ESWC 2018, Heraklion, Crete, Greece, June 3–7, 2018, Proceedings 15
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018 · 2018
Earlier work this paper cites.
Relational graph attention networks
Dan Busbridge, Dane Sherburn, Pietro Cavallo, and Nils Y Hammerla. 2019 · 2019
Earlier work this paper cites.
Roberta: A robustly optimized bert pretraining approach
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019 · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al · 2019
Earlier work this paper cites.
Heterogeneous graph attention network. In The world wide web conference
Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S Yu. 2019 · 2019
Earlier work this paper cites.
How Powerful are Graph Neural Networks?. In International Conference on Learning Representations
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2019 · 2019
Earlier work this paper cites.
Xlnet: Generalized autoregressive pretraining for language understanding. In Advances in neural information processing systems
Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019 · 2019
Earlier work this paper cites.
Graph convolutional networks for text classification. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence
Liang Yao, Chengsheng Mao, and Yuan Luo. 2019 · 2019
Earlier work this paper cites.
Language models are few-shot learners. In Advances in neural information processing systems
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 2020
Cited alongside, same era.
Pre-Trained Models for Heterogeneous Information Networks
Yang Fang, Xiang Zhao, Yifan Chen, Weidong Xiao, and Maarten de Rijke. 2020 · 2020
Cited alongside, same era.
Strategies for Pre-training Graph Neural Networks. In International Conference on Learning Representations
Weihua Hu*, Bowen Liu*, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. 2020 · 2020
Cited alongside, same era.
Heterogeneous graph transformer. In Proceedings of the web conference 2020
Ziniu Hu, Yuxiao Dong, Kuansan Wang, and Yizhou Sun. 2020a · 2020
Cited alongside, same era.
Self-supervised auxiliary learning with meta-paths for heterogeneous graphs. In Advances in Neural Information Processing Systems
JointGT: Graph-Text Joint Representation Learning for Text Generation from Knowledge Graphs. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021
Pei Ke, Haozhe Ji, Yu Ran, Xin Cui, Liwei Wang, Linfeng Song, Xiaoyan Zhu, and Minlie Huang. 2021 · 2021
Later among the works it cites.
Adsgnn: Behavior-graph augmented relevance modeling in sponsored search. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval
Chaozhuo Li, Bochen Pang, Yuming Liu, Hao Sun, Zheng Liu, Xing Xie, Tianqi Yang, Yanling Cui, Liangjie Zhang, and Qi Zhang. 2021 · 2021
Later among the works it cites.
Learning to pre-train graph neural networks. In Proceedings of the AAAI conference on artificial intelligence
Yuanfu Lu, Xunqiang Jiang, Yuan Fang, and Chuan Shi. 2021 · 2021
Later among the works it cites.
Recent advances in natural language processing via large pre-trained language models: A survey
Bonan Min, Hayley Ross, Elior Sulem, Amir Pouran Ben Veyseh, Thien Huu Nguyen, Oscar Sainz, Eneko Agirre, Ilana Heinz, and Dan Roth. 2021 · 2021
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Dasol Hwang, Jinyoung Park, Sunyoung Kwon, KyungMin Kim, Jung-Woo Ha, and Hyunwoo J Kim. 2020 · 2020
Cited alongside, same era.
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020 · 2020
Cited alongside, same era.
Gcc: Graph contrastive coding for graph neural network pre-training. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining
Jiezhong Qiu, Qibin Chen, Yuxiao Dong, Jing Zhang, Hongxia Yang, Ming Ding, Kuansan Wang, and Jie Tang. 2020 · 2020
Cited alongside, same era.
Exploring the limits of transfer learning with a unified text-to-text transformer
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020 · 2020
Cited alongside, same era.
Self-supervised graph transformer on large-scale molecular data. In Advances in Neural Information Processing Systems
Yu Rong, Yatao Bian, Tingyang Xu, Weiyang Xie, Ying Wei, Wenbing Huang, and Junzhou Huang. 2020 · 2020
Cited alongside, same era.
Knowledge-aware language model pretraining
Corby Rosset, Chenyan Xiong, Minh Phan, Xia Song, Paul Bennett, and Saurabh Tiwary. 2020 · 2020
Cited alongside, same era.
Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
Tao Shen, Yi Mao, Pengcheng He, Guodong Long, Adam Trischler, and Weizhu Chen. 2020 · 2020
Cited alongside, same era.
Heterogeneous network representation learning: A unified framework with survey and benchmark
Carl Yang, Yuxin Xiao, Yu Zhang, Yizhou Sun, and Jiawei Han. 2020 · 2020
Cited alongside, same era.
KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation
Xiaozhi Wang, Tianyu Gao, Zhaocheng Zhu, Zhengyan Zhang, Zhiyuan Liu, Juanzi Li, and Jian Tang. 2021 · 2021
Later among the works it cites.
Textgnn: Improving text encoder via graph neural network in sponsored search. In Proceedings of the Web Conference 2021
Jason Zhu, Yanling Cui, Yuming Liu, Hao Sun, Xue Li, Markus Pelger, Tianqi Yang, Liangjie Zhang, Ruofei Zhang, and Huasha Zhao. 2021a · 2021
Later among the works it cites.
Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Prediction. In International Conference on Learning Representations
Eli Chien, Wei-Cheng Chang, Cho-Jui Hsieh, Hsiang-Fu Yu, Jiong Zhang, Olgica Milenkovic, and Inderjit S Dhillon. 2022 · 2022
Later among the works it cites.
Efficient and effective training of language and graph neural network models
Vassilis N Ioannidis, Xiang Song, Da Zheng, Houyu Zhang, Jun Ma, Yi Xu, Belinda Zeng, Trishul Chilimbi, and George Karypis. 2022 · 2022
Later among the works it cites.
GNN-LM: Language Modeling based on Global Contexts via GNN. In International Conference on Learning Representations
Yuxian Meng, Shi Zong, Xiaoya Li, Xiaofei Sun, Tianwei Zhang, Fei Wu, and Jiwei Li. 2022 · 2022
Later among the works it cites.
Does GNN Pretraining Help Molecular Representation?. In Advances in Neural Information Processing Systems
Ruoxi Sun, Hanjun Dai, and Adams Wei Yu. 2022 · 2022
Later among the works it cites.
Deep Bidirectional Language-Knowledge Graph Pretraining. In Advances in Neural Information Processing Systems
Michihiro Yasunaga, Antoine Bosselut, Hongyu Ren, Xikun Zhang, Christopher D Manning, Percy Liang, and Jure Leskovec. 2022 · 2022
Later among the works it cites.
Jaket: Joint pre-training of knowledge graph and language understanding. In Proceedings of the AAAI Conference on Artificial Intelligence
Donghan Yu, Chenguang Zhu, Yiming Yang, and Michael Zeng. 2022 · 2022
Later among the works it cites.
OpenAI. 2023 · 2023
Closest in time.