Fetching the paper…
Reading the bibliography…
In multi-modal learning, some modalities are more influential than others, and their absence can have a significant impact on classification/segmentation accuracy.
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner · 1998
Earlier work this paper cites.
Rectified linear units improve restricted boltzmann machines
Vinod Nair and Geoffrey E Hinton · 2010
Earlier work this paper cites.
Autoencoders, unsupervised learning, and deep architectures
Pierre Baldi · 2012
Earlier work this paper cites.
The multimodal brain tumor image segmentation benchmark (brats)
Bjoern H Menze, Andras Jakab, Stefan Bauer, Jayashree Kalpathy-Cramer, Keyvan Farahani, Justin Kirby, Yuliya Burren, Nicole Porz, Johannes Slotboom, Roland Wiest, et al · 2014
Earlier work this paper cites.
Conditional generative adversarial nets
Mehdi Mirza and Simon Osindero · 2014
Earlier work this paper cites.
Learning structured output representation using deep conditional generative models
Kihyuk Sohn, Honglak Lee, and Xinchen Yan · 2015
Earlier work this paper cites.
Hemis: Hetero-modal image segmentation
Mohammad Havaei, Nicolas Guizard, Nicolas Chapados, and Yoshua Bengio · 2016
Earlier work this paper cites.
Sgdr: Stochastic gradient descent with warm restarts
Ilya Loshchilov and Frank Hutter · 2016
Earlier work this paper cites.
Nesterov’s accelerated gradient and momentum as approximations to regularised update descent
Aleksandar Botev, Guy Lever, and David Barber · 2017
Earlier work this paper cites.
Multimodal mr synthesis via modality-invariant latent representation
Agisilaos Chartsias, Thomas Joyce, Mario Valerio Giuffrida, and Sotirios A Tsaftaris · 2017
Earlier work this paper cites.
Unified subspace learning for incomplete and unlabeled multi-view data
Qiyue Yin, Shu Wu, and Liang Wang · 2017
Earlier work this paper cites.
Emotion recognition in speech using cross-modal transfer in the wild
Samuel Albanie, Arsha Nagrani, Andrea Vedaldi, and Andrew Zisserman · 2018
Earlier work this paper cites.
Spyridon Bakas, Mauricio Reyes, Andras Jakab, Stefan Bauer, Markus Rempfler, Alessandro Crimi, Russell Takeshi Shinohara, Christoph Berger, Sung Min Ha, Martin Rozycki, et al · 2018
Earlier work this paper cites.
Diverse image-to-image translation via disentangled representations
Hsin-Ying Lee, Hung-Yu Tseng, Jia-Bin Huang, Maneesh Singh, and Ming-Hsuan Yang · 2018
Earlier work this paper cites.
Learning cross-modality representations from multi-modal images
Gijs van Tulder and Marleen de Bruijne · 2018
Earlier work this paper cites.
Centralnet: a multilayer approach for multimodal fusion
Valentin Vielzeuf, Alexis Lechervy, Stéphane Pateux, and Frédéric Jurie · 2018
Earlier work this paper cites.
Robust multimodal brain tumor segmentation via feature disentanglement and gated fusion
Cheng Chen, Qi Dou, Yueming Jin, Hao Chen, Jing Qin, and Pheng-Ann Heng · 2019
Earlier work this paper cites.
Hetero-modal variational encoder-decoder for joint modality completion and segmentation
Reuben Dorent, Samuel Joutard, Marc Modat, Sébastien Ourselin, and Tom Vercauteren · 2019
Cited alongside, same era.
fmriprep: a robust preprocessing pipeline for functional mri
Oscar Esteban, Christopher J Markiewicz, Ross W Blair, Craig A Moodie, A Ilkay Isik, Asier Erramuzpe, James D Kent, Mathias Goncalves, Elizabeth DuPre, Madeleine Snyder, et al · 2019
Cited alongside, same era.
Brain tumor segmentation on mri with missing modalities
Yan Shen and Mingchen Gao · 2019
Cited alongside, same era.
Unpaired multi-modal segmentation via knowledge distillation
Qi Dou, Quande Liu, Pheng Ann Heng, and Ben Glocker · 2020
Cited alongside, same era.
Generative adversarial networks
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio · 2020
Cited alongside, same era.
Knowledge distillation from multi-modal to mono-modal segmentation networks
Smu-net: Style matching u-net for brain tumor segmentation with missing modalities
Reza Azad, Nika Khosravi, and Dorit Merhof · 2022
Later among the works it cites.
Learning disentangled representations in the imaging domain
Xiao Liu, Pedro Sanchez, Spyridon Thermos, Alison Q O’Neil, and Sotirios A Tsaftaris · 2022
Later among the works it cites.
Are multimodal transformers robust to missing modality?
Mengmeng Ma, Jian Ren, Long Zhao, Davide Testuggine, and Xi Peng · 2022
Later among the works it cites.
Multimodal contrastive learning with limoe: the language-image mixture of experts
Basil Mustafa, Carlos Riquelme, Joan Puigcerver, Rodolphe Jenatton, and Neil Houlsby · 2022
Later among the works it cites.
Uncertainty-aware multi-modal learning via cross-modal random network prediction
Hu Wang, Jianpeng Zhang, Yuanhong Chen, Congbo Ma, Jodie Avery, Louise Hull, and Gustavo Carneiro · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Minhao Hu, Matthis Maillard, Ya Zhang, Tommaso Ciceri, Giammarco La Barbera, Isabelle Bloch, and Pietro Gori · 2020
Cited alongside, same era.
Semi-supervised multi-view deep discriminant representation learning
Xiaodong Jia, Xiao-Yuan Jing, Xiaoke Zhu, Songcan Chen, Bo Du, Ziyun Cai, Zhenyu He, and Dong Yue · 2020
Cited alongside, same era.
Stochastic segmentation networks: Modelling spatially correlated aleatoric uncertainty
Miguel Monteiro, Loïc Le Folgoc, Daniel Coelho de Castro, Nick Pawlowski, Bernardo Marques, Konstantinos Kamnitsas, Mark van der Wilk, and Ben Glocker · 2020
Cited alongside, same era.
Multi-modal self-supervision from generalized data transformations
Mandela Patrick, Yuki M Asano, Polina Kuznetsova, Ruth Fong, Joao F Henriques, Geoffrey Zweig, and Andrea Vedaldi · 2020
Cited alongside, same era.
Localizing visual sounds the hard way
Honglie Chen, Weidi Xie, Triantafyllos Afouras, Arsha Nagrani, Andrea Vedaldi, and Andrew Zisserman · 2021
Cited alongside, same era.
Rfnet: Region-aware fusion network for incomplete multi-modal brain tumor segmentation
Yuhang Ding, Xin Yu, and Yi Yang · 2021
Cited alongside, same era.
Modality completion via gaussian process prior variational autoencoders for multi-modal glioma segmentation
Mohammad Hamghalam, Alejandro F Frangi, Baiying Lei, and Amber L Simpson · 2021
Cited alongside, same era.
Yao Zhang, Nanjun He, Jiawei Yang, Yuexiang Li, Dong Wei, Yawen Huang, Yang Zhang, Zhiqiang He, and Yefeng Zheng · 2022
Later among the works it cites.
Disentangle first, then distill: A unified framework for missing modality imputation and alzheimer’s disease diagnosis
Yuanyuan Chen, Yongsheng Pan, Yong Xia, and Yixuan Yuan · 2023
Later among the works it cites.
Enhancing modality-agnostic representations via meta-learning for brain tumor segmentation
Aishik Konwer, Xiaoling Hu, Joseph Bae, Xuan Xu, Chao Chen, and Prateek Prasanna · 2023
Later among the works it cites.
Multimodal prompting with missing modalities for visual recognition
Yi-Lun Lee, Yi-Hsuan Tsai, Wei-Chen Chiu, and Chen-Yu Lee · 2023
Later among the works it cites.
Missmodal: Increasing robustness to missing modality in multimodal sentiment analysis
Ronghao Lin and Haifeng Hu · 2023
Later among the works it cites.
Gpt-4 technical report, 2023
OpenAI · 2023
Later among the works it cites.
Multi-modal learning with missing modality via shared-specific feature modelling
Hu Wang, Yuanhong Chen, Congbo Ma, Jodie Avery, Louise Hull, and Gustavo Carneiro · 2023
Later among the works it cites.
Learning unified hyper-network for multi-modal mr image synthesis and tumor segmentation with missing modalities
Heran Yang, Jian Sun, and Zongben Xu · 2023
Later among the works it cites.
Modality translation-based multimodal sentiment analysis under uncertain missing modalities
Zhizhong Liu, Bin Zhou, Dianhui Chu, Yuhang Sun, and Lingqiang Meng · 2024
Closest in time.
Vl-mamba: Exploring state space models for multimodal learning
Yanyuan Qiao, Zheng Yu, Longteng Guo, Sihan Chen, Zijia Zhao, Mingzhen Sun, Qi Wu, and Jing Liu · 2024
Closest in time.
Flex-moe: Modeling arbitrary modality combination via the flexible mixture-of-experts
Sukwon Yun, Inyoung Choi, Jie Peng, Yangfan Wu, Jingxuan Bao, Qiyiwen Zhang, Jiayi Xin, Qi Long, and Tianlong Chen · 2024
Closest in time.
Generate, then retrieve: Addressing missing modalities in multimodal learning via generative ai and moe
Sukwon Yun, Jiayi Xin, Inyoung Choi, Jie Peng, Ying Ding, Qi Long, and Tianlong Chen · 2025
Closest in time.