Fetching the paper…
Reading the bibliography…
In recent years, pretraining models have made significant advancements in the fields of natural language processing (NLP), computer vision (CV), and life sciences.
“Language models are few-shot learners”
Tom Brown et al · 1901
Earlier work this paper cites.
“Merck molecular force field. I. Basis, form, scope, parameterization, and performance of MMFF94”
Thomas Halgren · 1996
Earlier work this paper cites.
“Extended-connectivity fingerprints”
David Rogers and Mathew Hahn · 2010
Earlier work this paper cites.
“The Floyd–Warshall algorithm on graphs with negative cycles”
Stefan Hougardy · 2010
Earlier work this paper cites.
“Adam: A method for stochastic optimization”
Diederik Kingma and Jimmy Ba · 2014
Earlier work this paper cites.
“Quantum chemistry structures and properties of 134 kilo molecules”
Raghunathan Ramakrishnan, Pavlo Dral, Matthias Rupp and O Von · 2014
Earlier work this paper cites.
“Better informed distance geometry: using what we know to improve conformation generation”
Sereina Riniker and Gregory Landrum · 2015
Earlier work this paper cites.
“Seq2seq fingerprint: An unsupervised deep molecular embedding for drug discovery”
Zheng Xu, Sheng Wang, Feiyun Zhu and Junzhou Huang · 2017
Earlier work this paper cites.
“Schnet: A continuous-filter convolutional neural network for modeling quantum interactions”
Kristof Schütt et al · 2017
Earlier work this paper cites.
“Decoupled weight decay regularization”
Ilya Loshchilov and Frank Hutter · 2017
Earlier work this paper cites.
Paulius Micikevicius et al · 2017
Earlier work this paper cites.
“Attention is all you need”
Ashish Vaswani et al · 2017
Earlier work this paper cites.
“How powerful are graph neural networks?”
Keyulu Xu, Weihua Hu, Jure Leskovec and Stefanie Jegelka · 2018
Earlier work this paper cites.
“MoleculeNet: a benchmark for molecular machine learning”
Zhenqin Wu et al · 2018
Earlier work this paper cites.
“Machine learning for molecular and materials science”
Keith Butler et al · 2018
Earlier work this paper cites.
“Machine learning models based on molecular fingerprints and an extreme gradient boosting method lead to the discovery of JAK2 inhibitors”
Minjian Yang et al · 2019
Earlier work this paper cites.
“Smiles-bert: large scale unsupervised pre-training for molecular property prediction”
Sheng Wang et al · 2019
Earlier work this paper cites.
“Learning continuous and data-driven molecular descriptors by translating equivalent chemical representations”
Robin Winter, Floriane Montanari, Frank Noé and Djork-Arné Clevert · 2019
Cited alongside, same era.
“Language models are unsupervised multitask learners”
Alec Radford et al · 2019
Cited alongside, same era.
“Cross-lingual language model pretraining”
Alexis Conneau and Guillaume Lample · 2019
Cited alongside, same era.
“Self-supervised graph transformer on large-scale molecular data”
Yu Rong et al · 2020
Cited alongside, same era.
“ZINC20—a free ultralarge-scale chemical database for ligand discovery”
John Irwin et al · 2020
Cited alongside, same era.
“Scaling laws for neural language models”
Jared Kaplan et al · 2020
Josh Achiam et al · 2023
Later among the works it cites.
“Llama: Open and efficient foundation language models”
Hugo Touvron et al · 2023
Later among the works it cites.
“Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks”
Zhe Chen et al · 2023
Later among the works it cites.
Jinze Bai et al · 2023
Later among the works it cites.
Albert Jiang et al · 2023
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
“Pre-training molecular graph representation with 3d geometry”
Shengchao Liu et al · 2021
Cited alongside, same era.
“Do transformers really perform badly for graph representation?”
Chengxuan Ying et al · 2021
Cited alongside, same era.
“Geometry-enhanced molecular representation learning for property prediction”
Xiaomin Fang et al · 2022
Cited alongside, same era.
“Molecular contrastive learning of representations via graph neural networks”
Yuyang Wang, Jianren Wang, Zhonglin Cao and Amir Barati · 2022
Cited alongside, same era.
“3d infomax improves gnns for molecular property prediction”
Hannes Stärk et al · 2022
Cited alongside, same era.
“Scaling vision transformers”
Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby and Lucas Beyer · 2022
Cited alongside, same era.
Later among the works it cites.
“Scaling vision transformers to 22 billion parameters”
Mostafa Dehghani et al · 2023
Later among the works it cites.
“UniMax: Fairer and More Effective Language Sampling for Large-Scale Multilingual Pretraining”
Hyung Chung et al · 2023
Later among the works it cites.
“Highly Accurate Quantum Chemical Property Prediction with Uni-Mol+”, 2023
Shuqi Lu et al · 2023
Later among the works it cites.
“DeepSeek LLM: Scaling Open-Source Language Models with Longtermism”
DeepSeek-AI · 2024
Closest in time.
“Visual instruction tuning”
Haotian Liu, Chunyuan Li, Qingyang Wu and Yong Lee · 2024
Closest in time.
“Uncovering neural scaling laws in molecular representation learning”
Dingshuo Chen et al · 2024
Closest in time.
“Sora: Creating video from text”, 2024
OpenAI · 2024
Closest in time.
“Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models”
Yixin Liu et al · 2024
Closest in time.
“Unraveling the Mystery of Scaling Laws: Part I”, 2024
Hui Su, Zhi Tian, Xiaoyu Shen and Xunliang Cai · 2024
Closest in time.
“Uni-Core, an efficient distributed PyTorch framework”, 2024
Uni-Core · 2024
Closest in time.
“PyTorch 2: Faster Machine Learning Through Dynamic Python Bytecode Transformation and Graph Compilation”
Jason Ansel et al · 2024
Closest in time.