Fetching the paper…
Reading the bibliography…
Foundation models of time series have not been fully developed due to the limited availability of time series corpora and the underexploration of scalable pre-training.
Forecasting sales by exponentially weighted moving averages
Winters, P. R · 1960
Earlier work this paper cites.
Stl: A seasonal-trend decomposition
Cleveland, R. B., Cleveland, W. S., McRae, J. E., and Terpenning, I · 1990
Earlier work this paper cites.
A neural probabilistic language model
Bengio, Y., Ducharme, R., and Vincent, P · 2000
Earlier work this paper cites.
Time series analysis by state space methods , volume 38
Durbin, J. and Koopman, S. J · 2012
Earlier work this paper cites.
Box and jenkins: time series analysis, forecasting and control
Box, G · 2013
Earlier work this paper cites.
Adam: A method for stochastic optimization
Kingma, D. P. and Ba, J · 2014
Earlier work this paper cites.
Time series analysis: forecasting and control
Box, G. E., Jenkins, G. M., Reinsel, G. C., and Ljung, G. M · 2015
Earlier work this paper cites.
Attention is all you need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I · 2017
Earlier work this paper cites.
Modeling long-and short-term temporal patterns with deep neural networks
Lai, G., Chang, W.-C., Yang, Y., and Liu, H · 2018
Earlier work this paper cites.
Zhu, Y., Kiros, R., Zemel, R., Salakhutdinov, R., Urtasun, R., Torralba, A., and Fidler, S · 2018
Earlier work this paper cites.
N-beats: Neural basis expansion analysis for interpretable time series forecasting
Oreshkin, B. N., Carpov, D., Chapados, N., and Bengio, Y · 2019
Earlier work this paper cites.
Pytorch: An imperative style, high-performance deep learning library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al · 2019
Earlier work this paper cites.
Language models are few-shot learners
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al · 2020
Earlier work this paper cites.
An image is worth 16x16 words: Transformers for image recognition at scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al · 2020
Earlier work this paper cites.
Scaling laws for neural language models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D · 2020
Earlier work this paper cites.
Reformer: The efficient transformer
Kitaev, N., Kaiser, Ł., and Levskaya, A · 2020
Cited alongside, same era.
The m4 competition: 100,000 time series and 61 forecasting methods
Makridakis, S., Spiliotis, E., and Assimakopoulos, V · 2020
Cited alongside, same era.
On the opportunities and risks of foundation models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al · 2021
Cited alongside, same era.
Lora: Low-rank adaptation of large language models
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W · 2021
Cited alongside, same era.
Learning transferable visual models from natural language supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al · 2021
Tempo: Prompt-based generative pre-trained transformer for time series forecasting
Cao, D., Jia, F., Arik, S. O., Pfister, T., Zheng, Y., Ye, W., and Liu, Y · 2023
Later among the works it cites.
Nhits: Neural hierarchical interpolation for time series forecasting
Challu, C., Olivares, K. G., Oreshkin, B. N., Ramirez, F. G., Canseco, M. M., and Dubrawski, A · 2023
Later among the works it cites.
Llm4ts: Two-stage fine-tuning for time-series forecasting with pre-trained llms
Chang, C., Peng, W.-C., and Chen, T.-F · 2023
Later among the works it cites.
Long-term forecasting with tide: Time-series dense encoder
Das, A., Kong, W., Leach, A., Sen, R., and Yu, R · 2023
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting
Wu, H., Xu, J., Wang, J., and Long, M · 2021
Cited alongside, same era.
Informer: Beyond efficient transformer for long sequence time-series forecasting
Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., and Zhang, W · 2021
Cited alongside, same era.
Flamingo: a visual language model for few-shot learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al · 2022
Cited alongside, same era.
Why can gpt learn in-context? language models secretly perform gradient descent as meta optimizers
Dai, D., Sun, Y., Dong, L., Hao, Y., Sui, Z., and Wei, F · 2022
Cited alongside, same era.
Training compute-optimal large language models
Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D. d. L., Hendricks, L. A., Welbl, J., Clark, A., et al · 2022
Cited alongside, same era.
Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation
Li, J., Li, D., Xiong, C., and Hoi, S · 2022
Cited alongside, same era.
Non-stationary transformers: Exploring the stationarity in time series forecasting
Liu, Y., Wu, H., Wang, J., and Long, M · 2022
Cited alongside, same era.
Gruver, N., Finzi, M., Qiu, S., and Wilson, A. G · 2023
Later among the works it cites.
Time-llm: Time series forecasting by reprogramming large language models
Jin, M., Wang, S., Ma, L., Chu, Z., Zhang, J. Y., Shi, X., Chen, P.-Y., Liang, Y., Li, Y.-F., Pan, S., et al · 2023
Later among the works it cites.
Gpt-4 technical report. arxiv 2303.08774
OpenAI, R · 2023
Later among the works it cites.
Test: Text prototype aligned embedding to activate llm’s ability for time series
Sun, C., Li, Y., Li, H., and Hong, S · 2023
Later among the works it cites.
Llama: Open and efficient foundation language models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al · 2023
Later among the works it cites.
Pushing the limits of pre-training for time series forecasting in the cloudops domain
Woo, G., Liu, C., Kumar, A., and Sahoo, D · 2023
Later among the works it cites.
Promptcast: A new prompt-based learning paradigm for time series forecasting
Xue, H. and Salim, F. D · 2023
Later among the works it cites.
Are transformers effective for time series forecasting?
Zeng, A., Chen, M., Zhang, L., and Xu, Q · 2023
Later among the works it cites.
A survey of large language models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., et al · 2023
Later among the works it cites.
One fits all: Power general time series analysis by pretrained lm
Zhou, T., Niu, P., Wang, X., Sun, L., and Jin, R · 2023
Later among the works it cites.
Roformer: Enhanced transformer with rotary position embedding
Su, J., Ahmed, M., Lu, Y., Pan, S., Bo, W., and Liu, Y · 2024
Closest in time.