Customize Segment Anything Model for Multi-Modal Semantic Segmentation with Mixture of LoRA Experts

H. Wu, J. Zhang, K. Huang, K. Liang, and Y. Yu, “Fastfcn: Rethinking dilated convolution in the backbone for semantic segmentation,” arXiv preprint arXiv:1903.11816 , Mar. 2019. [Online]. Available: http://arxiv.org/abs/1903.11816v1

Original

1903

Earlier work this paper cites.

M. Liu and H. Yin, “Feature pyramid encoding network for real-time semantic segmentation,” arXiv preprint arXiv:1909.08599 , Sep. 2019. [Online]. Available: http://arxiv.org/abs/1909.08599v1

Original

1909

Earlier work this paper cites.

J. Shi and J. Malik, “Normalized cuts and image segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 22, no. 8, pp. 888–905, 2000. [Online]. Available: https://doi.org/10.1109/34.868688

2000

Earlier work this paper cites.

O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015 . Springer International Publishing, 2015, pp. 234–241. [Online]. Available: https://doi.org/10.1007/978-3-319-24574-4\_28

2015

Earlier work this paper cites.

J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . IEEE, Jun. 2015, pp. 3431–3440. [Online]. Available: https://doi.org/10.1109/cvpr.2015.7298965

2015

Earlier work this paper cites.

V. Badrinarayanan, A. Kendall, and R. Cipolla, “Segnet: A deep convolutional encoder-decoder architecture for image segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 39, no. 12, pp. 2481–2495, Dec. 2017. [Online]. Available: https://doi.org/10.1109/tpami.2016.2644615

2016

Earlier work this paper cites.

A. Shrivastava, A. Gupta, and R. Girshick, “Training region-based object detectors with online hard example mining,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . IEEE, Jun. 2016, pp. 761–769. [Online]. Available: https://doi.org/10.1109/cvpr.2016.89

2016

Earlier work this paper cites.

L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 40, no. 4, pp. 834–848, Apr. 2018. [Online]. Available: https://doi.org/10.1109/tpami.2017.2699184

2017

Earlier work this paper cites.

C. Hazirbas, L. Ma, C. Domokos, and D. Cremers, “Fusenet: Incorporating depth into semantic segmentation via fusion-based cnn architecture,” in Computer Vision – ACCV 2016 . Springer International Publishing, 2017, pp. 213–228. [Online]. Available: https://doi.org/10.1007/978-3-319-54181-5\_14

2017

Earlier work this paper cites.

A. Valada, J. Vertens, A. Dhall, and W. Burgard, “Adapnet: Adaptive semantic segmentation in adverse environmental conditions,” in 2017 IEEE International Conference on Robotics and Automation (ICRA) , IEEE. IEEE, May 2017, pp. 4644–4651. [Online]. Available: https://doi.org/10.1109/icra.2017.7989540

2017

Earlier work this paper cites.

Y. Cheng, R. Cai, Z. Li, X. Zhao, and K. Huang, “Locality-sensitive deconvolution networks with gated fusion for rgb-d indoor semantic segmentation,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . IEEE, Jul. 2017, pp. 1475–1483. [Online]. Available: https://doi.org/10.1109/cvpr.2017.161

2017

Earlier work this paper cites.

X. Ding, Y. Guo, G. Ding, and J. Han, “Acnet: Strengthening the kernel skeletons for powerful cnn via asymmetric convolution blocks,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV) . IEEE, Oct. 2019, pp. 1911–1920. [Online]. Available: https://doi.org/10.1109/iccv.2019.00200

2019

Earlier work this paper cites.

Y. Sun, W. Zuo, and M. Liu, “Rtfnet: Rgb-thermal fusion network for semantic segmentation of urban scenes,” IEEE Robotics and Automation Letters , vol. 4, no. 3, pp. 2576–2583, Jul. 2019. [Online]. Available: https://doi.org/10.1109/lra.2019.2904733

2019

Earlier work this paper cites.

I. Loshchilov and F. Hutter, “Decoupled weight decay regularization.” in 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 . OpenReview.net, 2019. [Online]. Available: https://openreview.net/forum?id=Bkg6RiCqY7

2019

Earlier work this paper cites.

Y. Zhang, D. Sidibé, O. Morel, and F. Mériaudeau, “Deep multimodal fusion for semantic image segmentation: A survey,” Image and Vision Computing , vol. 105, p. 104042, Jan. 2021. [Online]. Available: https://doi.org/10.1016/j.imavis.2020.104042

2020

Earlier work this paper cites.

F. I. Diakogiannis, F. Waldner, P. Caccetta, and C. Wu, “Resunet-a: A deep learning framework for semantic segmentation of remotely sensed data,” ISPRS Journal of Photogrammetry and Remote Sensing , vol. 162, pp. 94–114, Apr. 2020. [Online]. Available: https://doi.org/10.1016/j.isprsjprs.2020.01.013

2020

Earlier work this paper cites.

S. Minaee, Y. Y. Boykov, F. Porikli, A. J. Plaza, N. Kehtarnavaz, and D. Terzopoulos, “Image segmentation using deep learning: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 44, no. 7, pp. 1–1, 2021. [Online]. Available: https://doi.org/10.1109/tpami.2021.3059968

2021

Earlier work this paper cites.

T. Tian, Z. Chu, Q. Hu, and L. Ma, “Class-wise fully convolutional network for semantic segmentation of remote sensing images,” Remote Sensing , vol. 13, no. 16, p. 3211, Aug. 2021. [Online]. Available: https://doi.org/10.3390/rs13163211

2021

Earlier work this paper cites.

S. Zheng, J. Lu, H. Zhao, X. Zhu, Z. Luo, Y. Wang, Y. Fu, J. Feng, T. Xiang, P. H. Torr, and L. Zhang, “Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers,” in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . IEEE, Jun. 2021, pp. 6881–6890. [Online]. Available: https://doi.org/10.1109/cvpr46437.2021.00681

2021

Earlier work this paper cites.

R. Strudel, R. Garcia, I. Laptev, and C. Schmid, “Segmenter: Transformer for semantic segmentation,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . IEEE, Oct. 2021, pp. 7262–7272. [Online]. Available: https://doi.org/10.1109/iccv48922.2021.00717

2021

Earlier work this paper cites.

J. Chen, Y. Lu, Q. Yu, X. Luo, E. Adeli, Y. Wang, L. Lu, A. L. Yuille, and Y. Zhou, “Transunet: Transformers make strong encoders for medical image segmentation,” arXiv preprint arXiv:2102.04306 , Feb. 2021. [Online]. Available: http://arxiv.org/abs/2102.04306v1

Original

2021

Earlier work this paper cites.

T. Panboonyuen, K. Jitkajornwanich, S. Lawawirojwong, P. Srestasathiern, and P. Vateekul, “Transformer-based decoder designs for semantic segmentation on remotely sensed images,” Remote Sensing , vol. 13, no. 24, p. 5100, Dec. 2021. [Online]. Available: https://doi.org/10.3390/rs13245100

2021

Earlier work this paper cites.

E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685 , Jun. 2021. [Online]. Available: http://arxiv.org/abs/2106.09685v2

Original

2021

Cited alongside, same era.

Y. Liang, R. Wakaki, S. Nobuhara, and K. Nishino, “Multimodal material segmentation,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . IEEE, Jun. 2022, pp. 19 768–19 776. [Online]. Available: https://doi.org/10.1109/cvpr52688.2022.01918

2022

Cited alongside, same era.

L. Wang, R. Li, C. Zhang, S. Fang, C. Duan, X. Meng, and P. M. Atkinson, “Unetformer: A unet-like transformer for efficient semantic segmentation of remote sensing urban scene imagery,” ISPRS Journal of Photogrammetry and Remote Sensing , vol. 190, pp. 196–214, Aug. 2022. [Online]. Available: https://doi.org/10.1016/j.isprsjprs.2022.06.008

2022

Cited alongside, same era.

N. Ding, X. Lv, Q. Wang, Y. Chen, B. Zhou, Z. Liu, and M. Sun, “Sparse low-rank adaptation of pre-trained language models,” arXiv preprint arXiv:2311.11696 , Nov. 2023. [Online]. Available: http://arxiv.org/abs/2311.11696v1

Original

2023

Later among the works it cites.

J.-Q. Jiang, G. Ye, and Y.-S. Piao, “Impact of the hubble tension on the r r - n ​ _ ​ s n\_s contour,” arXiv preprint arXiv:2303.12345 , Mar. 2023. [Online]. Available: http://arxiv.org/abs/2303.12345v2

Original

2023

Later among the works it cites.

C. Ryali, Y.-T. Hu, D. Bolya, C. Wei, H. Fan, P.-Y. Huang, V. Aggarwal, A. Chowdhury, O. Poursaeed, J. Hoffman et al. , “Hiera: A hierarchical vision transformer without the bells-and-whistles,” in International Conference on Machine Learning . PMLR, 2023, pp. 29 441–29 454

2023

Later among the works it cites.

H. Gu, H. Dong, J. Yang, and M. A. Mazurowski, “How to build the best medical image segmentation algorithm using foundation models: a comprehensive empirical study with segment anything model,” arXiv preprint arXiv:2404.09957 , Apr. 2024. [Online]. Available: http://arxiv.org/abs/2404.09957v2