Fetching the paper…
Reading the bibliography…
Multi-modal time series analysis has recently emerged as a prominent research area in data mining, driven by the increasing availability of diverse data modalities, such as text, images, and structured tabular data from real-world sources.
ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission
Kexin Huang, Jaan Altosaar, and Rajesh Ranganath. 2020 · 1904
Earlier work this paper cites.
VoxCeleb2: Deep Speaker Recognition. In Interspeech 2018 (Interspeech Proceedings) . ISCA
Joon Son Chung, Arsha Nagrani, and Andrew Zisserman. 2018 · 1929
Earlier work this paper cites.
Stock Movement Prediction from Tweets and Historical Prices. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , Iryna Gurevych and Yusuke Miyao (Eds.). Association for Computational Linguistics, Melbourne, Australia, 1970–1979
Yumo Xu and Shay B. Cohen. 2018 · 1979
Earlier work this paper cites.
Well googled is half done: Multimodal forecasting of new fashion product sales with image-based google trends
Geri Skenderi, Christian Joppi, Matteo Denitto, and Marco Cristani. 2024 · 1997
Earlier work this paper cites.
MIMIC-III, a freely accessible critical care database
Alistair E. W. Johnson, Tom J. Pollard, Lu Shen, Li-wei H. Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G. Mark. 2016 · 2016
Earlier work this paper cites.
A dual-stage attention-based recurrent neural network for time series prediction. In Proceedings of the 26th International Joint Conference on Artificial Intelligence . 2627–2633
Yao Qin, Dongjin Song, Haifeng Cheng, Wei Cheng, Guofei Jiang, and Garrison W Cottrell. 2017 · 2017
Earlier work this paper cites.
Stock price prediction via discovering multi-frequency trading patterns. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining . 2141–2149
Liheng Zhang, Charu Aggarwal, and Guo-Jun Qi. 2017 · 2017
Earlier work this paper cites.
LRS3-TED: a large-scale dataset for visual speech recognition
Triantafyllos Afouras, Joon Son Chung, and Andrew Zisserman. 2018 · 2018
Earlier work this paper cites.
Multimodal machine learning: A survey and taxonomy
Tadas Baltrušaitis, Chaitanya Ahuja, and Louis-Philippe Morency. 2018 · 2018
Earlier work this paper cites.
A treatment engine by predicting next-period prescriptions. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . 1608–1616
Bo Jin, Haoyu Yang, Leilei Sun, Chuanren Liu, Yue Qu, and Jianing Tong. 2018 · 2018
Earlier work this paper cites.
Hybrid Deep Sequential Modeling for Social Text-Driven Stock Prediction. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management (Torino, Italy) (CIKM ’18) . Association for Computing Machinery, New York, NY, USA, 1627–1630
Huizhe Wu, Wei Zhang, Weiwei Shen, and Jun Wang. 2018 · 2018
Earlier work this paper cites.
A data citation roadmap for scholarly data repositories
Helena Cousijn, Patricia Feeney, Daan Lowenberg, Elisa Presani, and Natasha Simons. 2019 · 2019
Earlier work this paper cites.
Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In Proceedings of the AAAI conference on artificial intelligence , Vol. 33. 922–929
Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. 2019 · 2019
Earlier work this paper cites.
Using Clinical Notes with Time Series Data for ICU Management. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) . Association for Computational Linguistics, Hong Kong, China, 6432–6437
Swaraj Khadanga, Karan Aggarwal, Shafiq Joty, and Jaideep Srivastava. 2019 · 2019
Earlier work this paper cites.
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan So, and Jaewoo Kang. 2019 · 2019
Earlier work this paper cites.
An open access database for the evaluation of respiratory sound classification algorithms
Bruno M Rocha, Dimitris Filos, Luís Mendes, Gorkem Serbes, Sezer Ulukaya, Yasemin P Kahya, Nikša Jakovljevic, Tatjana L Turukalo, Ioannis M Vogiatzis, Eleni Perantoni, Evangelos Kaimakamis, Pantelis Natsiavas, Ana Oliveira, Cristina Jácome, Alda Marques, Nicos Maglaveras, Rui Pedro Paiva, Ioanna Chouvarda, and Paulo de Carvalho. 2019 · 2019
Earlier work this paper cites.
Attention based multi-modal new product sales time-series forecasting. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining . 3110–3118
Vijay Ekambaram, Kushagra Manglik, Sumanta Mukherjee, Surya Shravan Kumar Sajja, Satyam Dwivedi, and Vikas Raykar. 2020 · 2020
Earlier work this paper cites.
ZuCo 2.0: A Dataset of Physiological Recordings During Natural Reading and Annotation. In Proceedings of the Twelfth Language Resources and Evaluation Conference , Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Asuncion Moreno, Jan Odijk, and Stelios Piperidis (Eds.). European Language Resources Association, Marseille, France, 138–146
Nora Hollenstein, Marius Troendle, Ce Zhang, and Nicolas Langer. 2020 · 2020
Earlier work this paper cites.
Deep Attentive Learning for Stock Movement Prediction From Social Media Text and Company Correlations. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, Online, 8415–8426
Ramit Sawhney, Shivam Agarwal, Arnav Wadhwa, and Rajiv Ratn Shah. 2020 · 2020
Earlier work this paper cites.
PTB-XL, a large publicly available electrocardiography dataset
Patrick Wagner, Nils Strodthoff, Ralf-Dieter Bousseljot, Dieter Kreiseler, Fatima I Lunze, Wojciech Samek, and Tobias Schaeffter. 2020 · 2020
Earlier work this paper cites.
Connecting the dots: Multivariate time series forecasting with graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . 753–763
Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. 2020 · 2020
Earlier work this paper cites.
Adversarial representation learning for robust patient-independent epileptic seizure detection
Xiang Zhang, Lina Yao, Manqing Dong, Zhe Liu, Yu Zhang, and Yong Li. 2020 · 2020
Earlier work this paper cites.
Predicting in-hospital mortality by combining clinical notes with time-series data. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 , Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (Eds.). Association for Computational Linguistics, Online, 4026–4031
Iman Deznabi, Mohit Iyyer, and Madalina Fiterau. 2021 · 2021
Earlier work this paper cites.
A dataset of lung sounds recorded from the chest wall using an electronic stethoscope
Mohammad Fraiwan, Luay Fraiwan, Basheer Khassawneh, and Ali Ibnian. 2021 · 2021
Earlier work this paper cites.
MIMIC-IV (version 1.0)
Alistair Johnson, Lucas Bulgarelli, Tom Pollard, Steven Horng, Leo Anthony Celi, and Roger Mark. 2021 · 2021
Earlier work this paper cites.
Stock price prediction using deep learning and frequency decomposition
Hadi Rezaei, Hamidreza Faaljou, and Gholamreza Mansourfar. 2021 · 2021
Earlier work this paper cites.
Open Vocabulary Electroencephalography-To-Text Decoding and Zero-shot Sentiment Classification. In AAAI Conference on Artificial Intelligence
Zhenhailong Wang and Heng Ji. 2021 · 2021
Earlier work this paper cites.
How to Leverage the Multimodal EHR Data for Better Medical Prediction?. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP) . Association for Computational Linguistics, 4029–4038
Bo Yang and Lijun Wu. 2021 · 2021
Earlier work this paper cites.
Multimodal Temporal-Clinical Note Network for Mortality Prediction
Haiyang Yang, Li Kuang, and FengQiang Xia. 2021 · 2021
Earlier work this paper cites.
Traffic flow forecasting with spatial-temporal graph diffusion network. In Proceedings of the AAAI conference on artificial intelligence , Vol. 35. 15008–15015
Xiyue Zhang, Chao Huang, Yong Xu, Lianghao Xia, Peng Dai, Liefeng Bo, Junbo Zhang, and Yu Zheng. 2021 · 2021
Earlier work this paper cites.
Physics-guided graph meta learning for predicting water temperature and streamflow in stream networks. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining . 2752–2761
Shengyu Chen, Jacob A Zwart, and Xiaowei Jia. 2022 · 2022
Earlier work this paper cites.
A large and rich EEG dataset for modeling human visual object recognition
Alessandro T. Gifford, Kshitij Dwivedi, Gemma Roig, and Radoslaw M. Cichy. 2022 · 2022
Earlier work this paper cites.
Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Prediction. In International Conference on Learning Representations
Bowen Shi, Wei-Ning Hsu, Kushal Lakhotia, and Abdelrahman Mohamed. 2022 · 2022
Cited alongside, same era.
Chain-of-thought prompting elicits reasoning in large language models (NIPS ’22) . Curran Associates Inc., Red Hook, NY, USA, Article 1800, 14 pages
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022 · 2022
Cited alongside, same era.
Multi-modal information analysis for fault diagnosis with time-series data from power transformer
Zhikai Xing and Yigang He. 2023 · 2022
Cited alongside, same era.
VIMTS: Variational-based Imputation for Multi-modal Time Series. In 2022 IEEE International Conference on Big Data (Big Data) . IEEE, 349–358
Xiaohu Zhao, Kebin Jia, Benjamin Letcher, Jennifer Fair, Yiqun Xie, and Xiaowei Jia. 2022 · 2022
Cited alongside, same era.
Multi-Modal Forecaster: Jointly Predicting Time Series and Textual Data
Kai Kim, Howard Tsai, Rajat Sen, Abhimanyu Das, Zihao Zhou, Abhishek Tanpure, Mathew Luo, and Rose Yu. 2024 · 2024
Later among the works it cites.
LEMMA-RCA: A Large Multi-modal Multi-domain Dataset for Root Cause Analysis
Dongjie Wang Chengyuan Deng Reon Matsuoka Lecheng Zheng, Zhengzhang Chen and Haifeng Chen. 2024 · 2024
Later among the works it cites.
MoAT: Multi-Modal Augmented Time Series Forecasting
Geon Lee, Wenchao Yu, Wei Cheng, and Haifeng Chen. 2024 · 2024
Later among the works it cites.
LITE: Modeling Environmental Ecosystems with Multimodal Large Language Models
Haoran Li, Junqi Liu, Zexian Wang, Shiyuan Luo, Xiaowei Jia, and Huaxiu Yao. 2024a · 2024
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Multi-Modal Financial Time-Series Retrieval Through Latent Space Projections. In Proceedings of the Fourth ACM International Conference on AI in Finance (Brooklyn, NY, USA) (ICAIF ’23) . Association for Computing Machinery, New York, NY, USA, 498–506
Tom Bamford, Andrea Coletta, Elizabeth Fons, Sriram Gopalakrishnan, Svitlana Vyetrenko, Tucker Balch, and Manuela Veloso. 2023 · 2023
Cited alongside, same era.
Coswara: a respiratory sounds and symptoms dataset for remote screening of SARS-CoV-2 infection
Dhananjay Bhattacharya, Nayan K Sharma, Debottam Dutta, Srikanth R Chetupalli, Prashant Mote, Sriram Ganapathy, Jyothi Bhat, Shreyas Ramoji, Pravin Ghosh, Aswin Subramanian, et al · 2023
Cited alongside, same era.
Physics-guided meta-learning method in baseflow prediction over large regions. In Proceedings of the 2023 SIAM International Conference on Data Mining (SDM) . SIAM, 217–225
Shengyu Chen, Yiqun Xie, Xiang Li, Xu Liang, and Xiaowei Jia. 2023b · 2023
Cited alongside, same era.
TSMixer: An All-MLP Architecture for Time Series Forecast-ing
Si-An Chen, Chun-Liang Li, Sercan O Arik, Nathanael Christian Yoder, and Tomas Pfister. 2023a · 2023
Cited alongside, same era.
ChatGPT Informed Graph Neural Network for Stock Movement Prediction
Zihan Chen, Lei Nico Zheng, Cheng Lu, Jialu Yuan, and Di Zhu. 2023c · 2023
Cited alongside, same era.
Best of both worlds: Multimodal contrastive learning with tabular and imaging data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 23924–23935
Paul Hager, Martin J Menten, and Daniel Rueckert. 2023 · 2023
Cited alongside, same era.
Spatio-temporal self-supervised learning for traffic flow prediction. In Proceedings of the AAAI conference on artificial intelligence , Vol. 37. 4356–4364
Jiahao Ji, Jingyuan Wang, Chao Huang, Junjie Wu, Boren Xu, Zhenhe Wu, Junbo Zhang, and Yu Zheng. 2023 · 2023
Cited alongside, same era.
Time series as images: Vision transformer for irregularly sampled time series
Zekun Li, Shiyang Li, and Xifeng Yan. 2023a · 2023
Cited alongside, same era.
Zhonghang Li, Lianghao Xia, Jiabin Tang, Yong Xu, Lei Shi, Long Xia, Dawei Yin, and Chao Huang. 2024b · 2024
Later among the works it cites.
Foundations & trends in multimodal machine learning: Principles, challenges, and open questions
Paul Pu Liang, Amir Zadeh, and Louis-Philippe Morency. 2024 · 2024
Later among the works it cites.
Decoding Time Series with LLMs: A Multi-Agent Framework for Cross-Domain Annotation
Minhua Lin, Zhengzhang Chen, Yanchi Liu, Xujiang Zhao, Zongyu Wu, Junxiang Wang, Xiang Zhang, Suhang Wang, and Haifeng Chen. 2024 · 2024
Later among the works it cites.
EEG2Text: Open Vocabulary EEG-to-Text Translation with Multi-View Transformer . In 2024 IEEE International Conference on Big Data (BigData) . IEEE Computer Society, Los Alamitos, CA, USA, 1824–1833
Hanwen Liu, Daniel Hajialigol, Benny Antony, Aiguo Han, and Xuan Wang. 2024a · 2024
Later among the works it cites.
How can large language models understand spatial-temporal data?
Lei Liu, Shuo Yu, Runze Wang, Zhenxun Ma, and Yanming Shen. 2024d · 2024
Later among the works it cites.
UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecasting. In Proceedings of the ACM Web Conference 2024 (Singapore, Singapore) (WWW ’24) . Association for Computing Machinery, New York, NY, USA, 4095–4106
Xu Liu, Junfeng Hu, Yuan Li, Shizhe Diao, Yuxuan Liang, Bryan Hooi, and Roger Zimmermann. 2024b · 2024
Later among the works it cites.
S 2 S^{2} IP-LLM: Semantic Space Informed Prompt Learning with LLM for Time Series Forecasting. In Forty-first International Conference on Machine Learning
Zijie Pan, Yushan Jiang, Sahil Garg, Anderson Schneider, Yuriy Nevmyvaka, and Dongjin Song. 2024 · 2024
Later among the works it cites.
On the Feasibility of Vision-Language Models for Time-Series Classification
Vinay Prithyani, Mohsin Mohammed, Richa Gadgil, Ricardo Buitrago, Vinija Jain, and Aman Chadha. 2024 · 2024
Later among the works it cites.
Exploring Multi-Modal Integration with Tool-Augmented LLM Agents for Precise Causal Discovery
ChengAo Shen, Zhengzhang Chen, Dongsheng Luo, Dongkuan Xu, Haifeng Chen, and Jingchao Ni. 2024 · 2024
Later among the works it cites.
From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflection. In Advances in Neural Information Processing Systems , A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37. Curran Associates, Inc., 58118–58153
Xinlei Wang, Maike Feng, Jing Qiu, JINJIN GU, and Junhua Zhao. 2024 · 2024
Later among the works it cites.
Context is key: A benchmark for forecasting with essential textual information
Andrew Robert Williams, Arjun Ashok, Étienne Marcotte, Valentina Zantedeschi, Jithendaraa Subramanian, Roland Riachi, James Requeima, Alexandre Lacoste, Irina Rish, Nicolas Chapados, et al · 2024
Later among the works it cites.
Frequency-domain MLPs are more effective learners in time series forecasting
Kun Yi, Qi Zhang, Wei Fan, Shoujin Wang, Pengyang Wang, Hui He, Ning An, Defu Lian, Longbing Cao, and Zhendong Niu. 2024 · 2024
Later among the works it cites.
Vision-language models for vision tasks: A survey
Jingyi Zhang, Jiaxing Huang, Sheng Jin, and Shijian Lu. 2024a · 2024
Later among the works it cites.
MULAN: multi-modal causal structure learning and root cause analysis for microservice systems. In Proceedings of the ACM Web Conference 2024 . 4107–4116
Lecheng Zheng, Zhengzhang Chen, Jingrui He, and Haifeng Chen. 2024 · 2024
Later among the works it cites.
Can LLMs Understand Time Series Anomalies?
Zihao Zhou and Rose Yu. 2024 · 2024
Later among the works it cites.
See it, Think it, Sorted: Large Multimodal Models are Few-shot Time Series Anomaly Analyzers
Jiaxin Zhuang, Leon Yan, Zhenwei Zhang, Ruiqi Wang, Jiawei Zhang, and Yuantao Gu. 2024 · 2024
Later among the works it cites.
Context Matters: Leveraging Contextual Features for Time Series Forecasting
Sameep Chattopadhyay, Pulkit Paliwal, Sai Shankar Narasimhan, Shubhankar Agarwal, and Sandeep P. Chinchali. 2025 · 2025
Closest in time.
InstrucTime: Advancing Time Series Classification with Multimodal Language Modeling. In Proceedings of the Eighteenth ACM International Conference on Web Search and Data Mining (Hannover, Germany) (WSDM ’25) . Association for Computing Machinery, New York, NY, USA, 792–800
Mingyue Cheng, Yiheng Chen, Qi Liu, Zhiding Liu, Yucong Luo, and Enhong Chen. 2025 · 2025
Closest in time.
Shi Bin Hoo, Samuel Müller, David Salinas, and Frank Hutter. 2025 · 2025
Closest in time.
Explainable Multi-modal Time Series Prediction with LLM-in-the-Loop
Yushan Jiang, Wenchao Yu, Geon Lee, Dongjin Song, Kijung Shin, Wei Cheng, Yanchi Liu, and Haifeng Chen. 2025 · 2025
Closest in time.
Position: Empowering Time Series Reasoning with Multimodal LLMs
Yaxuan Kong, Yiyuan Yang, Shiyu Wang, Chenghao Liu, Yuxuan Liang, Ming Jin, Stefan Zohren, Dan Pei, Yan Liu, and Qingsong Wen. 2025b · 2025
Closest in time.
TimeCAP: Learning to Contextualize, Augment, and Predict Time Series Events with Large Language Model Agents. In AAAI
Geon Lee, Wenchao Yu, Kijung Shin, Wei Cheng, and Haifeng Chen. 2025 · 2025
Closest in time.
Language in the Flow of Time: Time-Series-Paired Texts Weaved into a Unified Temporal Narrative
Zihao Li, Xiao Lin, Zhining Liu, Jiaru Zou, Ziwei Wu, Lecheng Zheng, Dongqi Fu, Yada Zhu, Hendrik Hamann, Hanghang Tong, and Jingrui He. 2025 · 2025
Closest in time.
TimeCMA: Towards LLM-Empowered Multivariate Time Series Forecasting via Cross-Modality Alignment. In AAAI
Chenxi Liu, Qianxiong Xu, Hao Miao, Sun Yang, Lingzheng Zhang, Cheng Long, Ziyue Li, and Rui Zhao. 2025 · 2025
Closest in time.
Harnessing Vision Models for Time Series Analysis: A Survey
Jingchao Ni, Ziming Zhao, ChengAo Shen, Hanghang Tong, Dongjin Song, Wei Cheng, Dongsheng Luo, and Haifeng Chen. 2025 · 2025
Closest in time.
Kanghui Ning, Zijie Pan, Yu Liu, Yushan Jiang, James Y. Zhang, Kashif Rasul, Anderson Schneider, Lintao Ma, Yuriy Nevmyvaka, and Dongjin Song. 2025 · 2025
Closest in time.
Jiahao Wang, Mingyue Cheng, Qingyang Mao, Yitong Zhou, Feiyang Xu, and Xin Li. 2025 · 2025
Closest in time.
Time-VLM: Exploring Multimodal Vision-Language Models for Augmented Time Series Forecasting
Siru Zhong, Weilin Ruan, Ming Jin, Huan Li, Qingsong Wen, and Yuxuan Liang. 2025 · 2025
Closest in time.