Fetching the paper…
Reading the bibliography…
Although we have witnessed great success of pre-trained models in natural language processing (NLP) and computer vision (CV), limited progress has been made for general time series analysis.
The interpolation of time series by related series
Friedman, M · 1962
Earlier work this paper cites.
Some recent advances in forecasting and control
Box, G. E. and Jenkins, G. M · 1968
Earlier work this paper cites.
Distribution of residual autocorrelations in autoregressive-integrated moving average time series models
Box, G. E. and Pierce, D. A · 1970
Earlier work this paper cites.
Long short-term memory
Hochreiter, S. and Schmidhuber, J · 1997
Earlier work this paper cites.
Timing of seasonal sales
Courty, P. and Li, H · 1999
Earlier work this paper cites.
Lacoste-Julien, S., Schmidt, M., and Bach, F · 2012
Earlier work this paper cites.
Empirical evaluation of gated recurrent neural networks on sequence modeling
Chung, J., Gulcehre, C., Cho, K., and Bengio, Y · 2014
Earlier work this paper cites.
Xgboost: A scalable tree boosting system
Chen, T. and Guestrin, C · 2016
Earlier work this paper cites.
Swat: A water treatment testbed for research and training on ics security
Mathur, A. P. and Tippenhauer, N. O · 2016
Earlier work this paper cites.
Probabilistic demand forecasting at scale
Böse, J.-H., Flunkert, V., Gasthaus, J., Januschowski, T., Lange, D., Salinas, D., Schelter, S., Seeger, M., and Wang, Y · 2017
Earlier work this paper cites.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser Lukasz, and Polosukhin, I · 2017
Earlier work this paper cites.
The uea multivariate time series classification archive, 2018
Bagnall, A., Dau, H. A., Lines, J., Flynn, M., Large, J., Bostrom, A., Southam, P., and Keogh, E · 2018
Earlier work this paper cites.
Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding
Hundman, K., Constantinou, V., Laporte, C., Colwell, I., and Soderstrom, T · 2018
Earlier work this paper cites.
Modeling long-and short-term temporal patterns with deep neural networks
Lai, G., Chang, W.-C., Yang, Y., and Liu, H · 2018
Earlier work this paper cites.
The m4 competition: Results, findings, conclusion and way forward
Makridakis, S., Spiliotis, E., and Assimakopoulos, V · 2018
Earlier work this paper cites.
Improving language understanding by generative pre-training
Radford, A. and Narasimhan, K · 2018
Earlier work this paper cites.
BERT: pre-training of deep bidirectional transformers for language understanding
Devlin, J., Chang, M., Lee, K., and Toutanova, K · 2019
Earlier work this paper cites.
Unsupervised scalable representation learning for multivariate time series
Franceschi, J.-Y., Dieuleveut, A., and Jaggi, M · 2019
Earlier work this paper cites.
Parameter-efficient transfer learning for nlp
Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., De Laroussilhe, Q., Gesmundo, A., Attariyan, M., and Gelly, S · 2019
Earlier work this paper cites.
Deep learning for time series classification: a review
Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., and Muller, P.-A · 2019
Earlier work this paper cites.
N-beats: Neural basis expansion analysis for interpretable time series forecasting
Oreshkin, B. N., Carpov, D., Chapados, N., and Bengio, Y · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I · 2019
Earlier work this paper cites.
Robust anomaly detection for multivariate time series through stochastic recurrent neural network
Su, Y., Zhao, Y., Niu, C., Liu, R., Sun, W., and Pei, D · 2019
Cited alongside, same era.
Language models are few-shot learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T. J., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D · 2020
Cited alongside, same era.
ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernels
Dempster, A., Petitjean, F., and Webb, G. I · 2020
Cited alongside, same era.
RobustTAD: Robust time series anomaly detection via decomposition and convolutional neural networks
Gao, J., Song, X., Wen, Q., Wang, P., Sun, L., and Xu, H · 2020
Cited alongside, same era.
Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting
Wu, H., Xu, J., Wang, J., and Long, M · 2021
Later among the works it cites.
Anomaly transformer: Time series anomaly detection with association discrepancy
Xu, J., Wu, H., Wang, J., and Long, M · 2021
Later among the works it cites.
Voice2series: Reprogramming acoustic models for time series classification
Yang, C.-H. H., Tsai, Y.-Y., and Chen, P.-Y · 2021
Later among the works it cites.
Informer: Beyond efficient transformer for long sequence time-series forecasting
Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., and Zhang, W · 2021
Later among the works it cites.
BEit: BERT pre-training of image transformers
Bao, H., Dong, L., Piao, S., and Wei, F · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Kitaev, N., Kaiser, L., and Levskaya, A · 2020
Cited alongside, same era.
Wang, T. and Isola, P · 2020
Cited alongside, same era.
Transformers: State-of-the-art natural language processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Scao, T. L., Gugger, S., Drame, M., Lhoest, Q., and Rush, A. M · 2020
Cited alongside, same era.
O (n) connections are expressive enough: Universal approximability of sparse transformers
Yun, C., Chang, Y.-W., Bhojanapalli, S., Rawat, A. S., Reddi, S., and Kumar, S · 2020
Cited alongside, same era.
Practical approach to asynchronous multivariate time series anomaly detection and localization
Abdulaal, A., Liu, Z., and Lancewicki, T · 2021
Cited alongside, same era.
Vlmo: Unified vision-language pre-training with mixture-of-modality-experts
Bao, H., Wang, W., Dong, L., Liu, Q., Mohammed, O. K., Aggarwal, K., Som, S., and Wei, F · 2021
Cited alongside, same era.
An image is worth 16x16 words: Transformers for image recognition at scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N · 2021
Cited alongside, same era.
A mathematical framework for transformer circuits
Elhage, N., Nanda, N., Olsson, C., Henighan, T., Joseph, N., Mann, B., Askell, A., Bai, Y., Chen, A., Conerly, T., DasSarma, N., Drain, D., Ganguli, D., Hatfield-Dodds, Z., Hernandez, D., Jones, A., Kernion, J., Lovitt, L., Ndousse, K., Amodei, D., Brown, T., Clark, J., Kaplan, J., McCandlish, S., and Olah, C · 2021
Cited alongside, same era.
Challu, C., Olivares, K. G., Oreshkin, B. N., Garza, F., Mergenthaler, M., and Dubrawski, A · 2022
Later among the works it cites.
Flowformer: A transformer architecture for optical flow
Huang, Z., Shi, X., Zhang, C., Wang, Q., Cheung, K. C., Qin, H., Dai, J., and Li, H · 2022
Later among the works it cites.
Reversible instance normalization for accurate time-series forecasting against distribution shift
Kim, T., Kim, J., Tae, Y., Park, C., Choi, J.-H., and Choo, J · 2022
Later among the works it cites.
Non-stationary transformers: Exploring the stationarity in time series forecasting
Liu, Y., Wu, H., Wang, J., and Long, M · 2022
Later among the works it cites.
Frozen pretrained transformers as universal computation engines
Lu, K., Grover, A., Abbeel, P., and Mordatch, I · 2022
Later among the works it cites.
A time series is worth 64 words: Long-term forecasting with transformers
Nie, Y., Nguyen, N. H., Sinthong, P., and Kalagnanam, J · 2022
Later among the works it cites.
In-context learning and induction heads
Olsson, C., Elhage, N., Nanda, N., Joseph, N., DasSarma, N., Henighan, T., Mann, B., Askell, A., Bai, Y., Chen, A., Conerly, T., Drain, D., Ganguli, D., Hatfield-Dodds, Z., Hernandez, D., Johnston, S., Jones, A., Kernion, J., Lovitt, L., Ndousse, K., Amodei, D., Brown, T., Clark, J., Kaplan, J., McCandlish, S., and Olah, C · 2022
Later among the works it cites.
Kvt: k-nn attention for boosting vision transformers
Wang, P., Wang, X., Wang, F., Lin, M., Chang, S., Li, H., and Jin, R · 2022
Later among the works it cites.
Robust time series analysis and applications: An industrial perspective
Wen, Q., Yang, L., Zhou, T., and Sun, L · 2022
Later among the works it cites.
Etsformer: Exponential smoothing transformers for time-series forecasting
Woo, G., Liu, C., Sahoo, D., Kumar, A., and Hoi, S · 2022
Later among the works it cites.
Less is more: Fast multivariate time series forecasting with light sampling-oriented mlp structures
Zhang, T., Zhang, Y., Cao, W., Bian, J., Yi, X., Zheng, S., and Li, J · 2022
Later among the works it cites.
FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting
Zhou, T., Ma, Z., Wen, Q., Wang, X., Sun, L., and Jin, R · 2022
Later among the works it cites.
Looped Transformers as Programmable Computers
Giannou, A., Rajput, S., Sohn, J.-y., Lee, K., Lee, J. D., and Papailiopoulos, D · 2023
Closest in time.
OpenAI · 2023
Closest in time.
Transformers in time series: A survey
Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., and Sun, L · 2023
Closest in time.
Timesnet: Temporal 2d-variation modeling for general time series analysis
Wu, H., Hu, T., Liu, Y., Zhou, H., Wang, J., and Long, M · 2023
Closest in time.
Are transformers effective for time series forecasting?
Zeng, A., Chen, M., Zhang, L., and Xu, Q · 2023
Closest in time.