Fetching the paper…
Reading the bibliography…
We introduce EarthPT -- an Earth Observation (EO) pretrained transformer.
“LIII. On lines and planes of closest fit to systems of points in space”
K. Pearson · 1901
Earlier work this paper cites.
“Quantifying the Carbon Emissions of Machine Learning”
A. Lacoste, A. Luccioni, V. Schmidt and T. Dandres · 1910
Earlier work this paper cites.
“Robust Estimation of a Location Parameter”
P.. Huber · 1964
Earlier work this paper cites.
“Learning Curves: Asymptotic Values and Rate of Convergence”
C. Cortes et al · 1993
Earlier work this paper cites.
“Scaling Laws for Neural Language Models”
J. Kaplan et al · 2001
Earlier work this paper cites.
“Adam: A Method for Stochastic Optimization”
D.. Kingma and J. Ba · 2015
Earlier work this paper cites.
“BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”
J. Devlin, M. Chang, K. Lee and K. Toutanova · 2019
Earlier work this paper cites.
“Language Models are Unsupervised Multitask Learners”
A. Radford et al · 2019
Earlier work this paper cites.
“Language Models are Few-Shot Learners”
T. Brown et al · 2020
Earlier work this paper cites.
“Satellite Image Time Series Classification With Pixel-Set Encoders and Temporal Self-Attention”
V… Garnot, L. Landrieu, S. Giordano and N. Chehata · 2020
Earlier work this paper cites.
“Self-attention for raw optical Satellite Time Series Classification”
M. Russwurm and M. Körner · 2020
Cited alongside, same era.
“Array programming with NumPy”
C.. Harris et al · 2020
Cited alongside, same era.
“GPT-NeoX-20B: An Open-Source Autoregressive Language Model”
S. Black et al · 2022
Cited alongside, same era.
“Training Compute-Optimal Large Language Models”
J. Hoffmann et al · 2022
Cited alongside, same era.
“Chain-of-Thought Prompting Elicits Reasoning in Large Language Models”
Jason Wei et al · 2022
Cited alongside, same era.
“Chinchilla’s Wild Implications”, 2022
R. Friel · 2022
Cited alongside, same era.
“An Efficient Global Scale Sentinel-1 Radar Backscatter and Interferometric Processing System”
P.. Agram, M.. Warren, M.. Calef and S.. Arko · 2022
Later among the works it cites.
“Forecasting vegetation condition with a Bayesian auto-regressive distributed lags (BARDL) model”
E.. Salakpi et al · 2022
Later among the works it cites.
“A Generalist Agent”
S. Reed et al · 2022
Later among the works it cites.
“GPT-4 Technical Report”
OpenAI · 2023
Closest in time.
“RWKV: Reinventing RNNs for the Transformer Era”
B. Peng et al · 2023
Closest in time.
“Astronomia ex machina: a history, primer and outlook on neural networks in astronomy”
M.. Smith and J.. Geach · 2023
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
P. Villalobos et al · 2022
Cited alongside, same era.
“SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery”
Y. Cong et al · 2022
Cited alongside, same era.
“Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning”
C.. Reed et al · 2022
Cited alongside, same era.
“Lightweight, Pre-trained Transformers for Remote Sensing Timeseries”
G. Tseng et al · 2023
Closest in time.
“Llama 2: Open Foundation and Fine-Tuned Chat Models”
H. Touvron et al · 2023
Closest in time.
“Towards an astronomical foundation model for stars with a Transformer-based model”
H.. Leung and J. Bovy · 2023
Closest in time.