Fetching the paper…
Reading the bibliography…
We introduce MOMENT, a family of open-source foundation models for general-purpose time series analysis.
Climate modeling
Schneider, S. H. and Dickinson, R. E · 1974
Earlier work this paper cites.
Efficient algorithms for mining outliers from large data sets
Ramaswamy, S., Rastogi, R., and Shim, K · 2000
Earlier work this paper cites.
Accelerating t-sne using tree-based algorithms
van der Maaten, L · 2014
Earlier work this paper cites.
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A. C., and Fei-Fei, L · 2015
Earlier work this paper cites.
ElectricityLoadDiagrams20112014
Trindade, A · 2015
Earlier work this paper cites.
Layer normalization, 2016
Ba, J. L., Kiros, J. R., and Hinton, G. E · 2016
Earlier work this paper cites.
Multi-scale convolutional neural networks for time series classification, 2016
Cui, Z., Chen, W., and Chen, Y · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
He, K., Zhang, X., Ren, S., and Sun, J · 2016
Earlier work this paper cites.
Data Augmentation for Time Series Classification using Convolutional Neural Networks
Le Guennec, A., Malinowski, S., and Tavenard, R · 2016
Earlier work this paper cites.
Time series classification using time warping invariant echo state networks
Tanisaro, P. and Heidemann, G · 2016
Earlier work this paper cites.
Neural discrete representation learning
Van Den Oord, A., Vinyals, O., et al · 2017
Earlier work this paper cites.
Attention is all you need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L. u., and Polosukhin, I · 2017
Earlier work this paper cites.
Time series classification from scratch with deep neural networks: A strong baseline
Wang, Z., Yan, W., and Oates, T · 2017
Earlier work this paper cites.
Convolutional Neural Networks for Time Series Classification , pp. 635–642
Zebik, M., Korytkowski, M., Angryk, R., and Scherer, R · 2017
Earlier work this paper cites.
The ucr time series classification archive, October 2018
Dau, H. A., Keogh, E., Kamgar, K., Yeh, C.-C. M., Zhu, Y., Gharghabi, S., Ratanamahatana, C. A., Yanping, Hu, B., Begum, N., Bagnall, A., Mueen, A., Batista, G., and Hexagon-ML · 2018
Earlier work this paper cites.
Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding
Hundman, K., Constantinou, V., Laporte, C., Colwell, I., and Soderstrom, T · 2018
Earlier work this paper cites.
Modeling long- and short-term temporal patterns with deep neural networks
Lai, G., Chang, W.-C., Yang, Y., and Liu, H · 2018
Earlier work this paper cites.
Towards a universal neural network encoder for time series
Serrà, J., Pascual, S., and Karatzoglou, A · 2018
Earlier work this paper cites.
Self-attention with relative position representations
Shaw, P., Uszkoreit, J., and Vaswani, A · 2018
Earlier work this paper cites.
Unsupervised anomaly detection via variational auto-encoder for seasonal kpis in web applications
Xu, H., Chen, W., Zhao, N., Li, Z., Bu, J., Li, Z., Liu, Y., Zhao, Y., Pei, D., Feng, Y., et al · 2018
Earlier work this paper cites.
BERT: Pre-training of deep bidirectional transformers for language understanding
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K · 2019
Earlier work this paper cites.
Unsupervised scalable representation learning for multivariate time series
Franceschi, J.-Y., Dieuleveut, A., and Jaggi, M · 2019
Earlier work this paper cites.
Deep learning for time series classification: a review
Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., and Muller, P.-A · 2019
Earlier work this paper cites.
Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting
Li, S., Jin, X., Xuan, Y., Zhou, X., Chen, W., Wang, Y.-X., and Yan, X · 2019
Earlier work this paper cites.
Decoupled weight decay regularization
Loshchilov, I. and Hutter, F · 2019
Earlier work this paper cites.
Super-convergence: Very fast training of neural networks using large learning rates
Smith, L. N. and Topin, N · 2019
Earlier work this paper cites.
Robust anomaly detection for multivariate time series through stochastic recurrent neural network
Su, Y., Zhao, Y., Niu, C., Liu, R., Sun, W., and Pei, D · 2019
Earlier work this paper cites.
The pile: An 800gb dataset of diverse text for language modeling, 2020
Gao, L., Biderman, S., Black, S., Golding, L., Hoppe, T., Foster, C., Phang, J., He, H., Thite, A., Nabeshima, N., Presser, S., and Leahy, C · 2020
Earlier work this paper cites.
Meta-learning for few-shot time series classification
Narwariya, J., Malhotra, P., Vig, L., Shroff, G., and Vishnu, T. V · 2020
Earlier work this paper cites.
N-beats: Neural basis expansion analysis for interpretable time series forecasting
Oreshkin, B. N., Carpov, D., Chapados, N., and Bengio, Y · 2020
Cited alongside, same era.
Exploring the limits of transfer learning with a unified text-to-text transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J · 2020
Cited alongside, same era.
On the dangers of stochastic parrots: Can language models be too big?
Bender, E. M., Gebru, T., McMillan-Major, A., and Shmitchell, S · 2021
Cited alongside, same era.
An image is worth 16x16 words: Transformers for image recognition at scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N · 2021
Cited alongside, same era.
Time-series representation learning via temporal and contextual contrasting
Eldele, E., Ragab, M., Chen, Z., Wu, M., Kwoh, C. K., Li, X., and Guan, C · 2021
Cited alongside, same era.
FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting
Zhou, T., Ma, Z., Wen, Q., Wang, X., Sun, L., and Jin, R · 2022
Later among the works it cites.
The foundation model transparency index, 2023
Bommasani, R., Klyman, K., Longpre, S., Kapoor, S., Maslej, N., Xiong, B., Zhang, D., and Liang, P · 2023
Later among the works it cites.
Jolt: Jointly learned representations of language and time-series
Cai, Y., Goswami, M., Choudhry, A., Srinivasan, A., and Dubrawski, A · 2023
Later among the works it cites.
Tempo: Prompt-based generative pre-trained transformer for time series forecasting, 2023
Cao, D., Jia, F., Arik, S. O., Pfister, T., Zheng, Y., Ye, W., and Liu, Y · 2023
Later among the works it cites.
NHITS: Neural hierarchical interpolation for time series forecasting
Challu, C., Olivares, K. G., Oreshkin, B. N., Garza Ramirez, F., Mergenthaler Canseco, M., and Dubrawski, A · 2023
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Monash time series forecasting archive
Godahewa, R. W., Bergmeir, C., Webb, G. I., Hyndman, R., and Montero-Manso, P · 2021
Cited alongside, same era.
Weak supervision for affordable modeling of electrocardiogram data
Goswami, M., Boecking, B., and Dubrawski, A · 2021
Cited alongside, same era.
Meta-learning framework with applications to zero-shot time-series forecasting
Oreshkin, B. N., Carpov, D., Chapados, N., and Bengio, Y · 2021
Cited alongside, same era.
Carbon emissions and large neural network training, 2021
Patterson, D., Gonzalez, J., Le, Q., Liang, C., Munguia, L.-M., Rothchild, D., So, D., Texier, M., and Dean, J · 2021
Cited alongside, same era.
Learning transferable visual models from natural language supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I · 2021
Cited alongside, same era.
Mlp-mixer: An all-mlp architecture for vision
Tolstikhin, I. O., Houlsby, N., Kolesnikov, A., Beyer, L., Zhai, X., Unterthiner, T., Yung, J., Steiner, A., Keysers, D., Uszkoreit, J., et al · 2021
Cited alongside, same era.
Unsupervised representation learning for time series with temporal neighborhood coding
Tonekaboni, S., Eytan, D., and Goldenberg, A · 2021
Cited alongside, same era.
Das, A., Kong, W., Sen, R., and Zhou, Y · 2023
Later among the works it cites.
Video pre-trained transformer: A multimodal mixture of pre-trained experts, 2023
Day, K., Christl, D., Salvi, R., and Sriram, P · 2023
Later among the works it cites.
Simmtm: A simple pre-training framework for masked time-series modeling
Dong, J., Wu, H., Zhang, H., Zhang, L., Wang, J., and Long, M · 2023
Later among the works it cites.
Large language models are zero-shot time series forecasters
Gruver, N., Finzi, M. A., Qiu, S., and Wilson, A. G · 2023
Later among the works it cites.
Time-llm: Time series forecasting by reprogramming large language models, 2023
Jin, M., Wang, S., Ma, L., Chu, Z., Zhang, J. Y., Shi, X., Chen, P.-Y., Liang, Y., Li, Y.-F., Pan, S., and Wen, Q · 2023
Later among the works it cites.
Scaling language-image pre-training via masking
Li, Y., Fan, H., Hu, R., Feichtenhofer, C., and He, K · 2023
Later among the works it cites.
itransformer: Inverted transformers are effective for time series forecasting
Liu, Y., Hu, T., Zhang, H., Wu, H., Wang, S., Ma, L., and Long, M · 2023
Later among the works it cites.
A survey on time-series pre-trained models, 2023
Ma, Q., Liu, Z., Zheng, Z., Huang, Z., Zhu, S., Yu, Z., and Kwok, J. T · 2023
Later among the works it cites.
A time series is worth 64 words: Long-term forecasting with transformers
Nie, Y., Nguyen, N. H., Sinthong, P., and Kalagnanam, J · 2023
Later among the works it cites.
Lag-llama: Towards foundation models for time series forecasting
Rasul, K., Ashok, A., Williams, A. R., Khorasani, A., Adamopoulos, G., Bhagwatkar, R., Biloš, M., Ghonia, H., Hassen, N. V., Schneider, A., et al · 2023
Later among the works it cites.
Cross-modal fine-tuning: Align then refine, 2023
Shen, J., Li, L., Dery, L. M., Staten, C., Khodak, M., Neubig, G., and Talwalkar, A · 2023
Later among the works it cites.
Llama 2: Open foundation and fine-tuned chat models, 2023
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., Bikel, D., Blecher, L., Ferrer, C. C., Chen, M., Cucurull, G., Esiobu, D., Fernandes, J., Fu, J., Fu, W., Fuller, B., Gao, C., Goswami, V., Goyal, N., Hartshorn, A., Hosseini, S., Hou, R., Inan, H., Kardas, M., Kerkez, V., Khabsa, M., Kloumann, I., Korenev, A., Koura, P. S., Lachaux, M.-A., Lavril, T., Lee, J., Liskovich, D., Lu, Y., Mao, Y., Martinet, X., Mihaylov, T., Mishra, P., Molybog, I., Nie, Y., Poulton, A., Reizenstein, J., Rungta, R., Saladi, K., Schelten, A., Silva, R., Smith, E. M., Subramanian, R., Tan, X. E., Tang, B., Taylor, R., Williams, A., Kuan, J. X., Xu, P., Yan, Z., Zarov, I., Zhang, Y., Fan, A., Kambadur, M., Narang, S., Rodriguez, A., Stojnic, R., Edunov, S., and Scialom, T · 2023
Later among the works it cites.
Transformers in time series: A survey
Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., and Sun, L · 2023
Later among the works it cites.
Timesnet: Temporal 2d-variation modeling for general time series analysis
Wu, H., Hu, T., Liu, Y., Zhou, H., Wang, J., and Long, M · 2023
Later among the works it cites.
Chronos: Learning the language of time series
Ansari, A. F., Stella, L., Turkmen, C., Zhang, X., Mercado, P., Shen, H., Shchur, O., Rangapuram, S. S., Arango, S. P., Kapoor, S., et al · 2024
Closest in time.
Performance measurement system (pems), 2024
California Department of Transportation · 2024
Closest in time.
Fluview: Flu activity & surveillance, 2024
Centers for Disease Control and Prevention · 2024
Closest in time.
Tiny time mixers (ttms): Fast pre-trained models for enhanced zero/few-shot forecasting of multivariate time series, 2024
Ekambaram, V., Jati, A., Nguyen, N. H., Dayama, P., Reddy, C., Gifford, W. M., and Kalagnanam, J · 2024
Closest in time.
Timer: Generative pre-trained transformers are large time series models
Liu, Y., Zhang, H., Li, C., Huang, X., Wang, J., and Long, M · 2024
Closest in time.
Weather data, 2024
Max Planck Institute for Biogeochemistry · 2024
Closest in time.
Totem: Tokenized time series embeddings for general time series analysis, 2024
Talukder, S., Yue, Y., and Gkioxari, G · 2024
Closest in time.
Unified training of universal time series forecasting transformers
Woo, G., Liu, C., Kumar, A., Xiong, C., Savarese, S., and Sahoo, D · 2024
Closest in time.
One fits all: Power general time series analysis by pretrained LM
Zhou, T., Niu, P., Wang, X., Sun, L., and Jin, R · 2024
Closest in time.