Built on

Vaswani, A., Shazeer, N., Parmar, et al.: Attention is all you need. NIPS 30
2017
Earlier work this paper cites.
Devlin, J., Chang, M.W., Lee, et al.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Original
2018
Earlier work this paper cites.
Lau, J.J., Gayen, et al.: A dataset of clinically generated visual questions and answers about radiology images. Scientific data 5
2018
Earlier work this paper cites.
Houlsby, N., Giurgiu, A., Jastrzebski, et al.: Parameter-efficient transfer learning for nlp. In: International Conference on Machine Learning. pp. 2790–2799. PMLR (2019)
2019
Earlier work this paper cites.
Brown, T., Mann, B., Ryder, et al.: Language models are few-shot learners. Advances in neural information processing systems 33
2020
Earlier work this paper cites.
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)
Original
2020
Earlier work this paper cites.
Hu, E.J., Shen, et al.: Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021)
Original
2021
Earlier work this paper cites.
Lester, B., Al-Rfou, R., Constant, N.: The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691 (2021)
Original
2021
Earlier work this paper cites.
Li, X.L., Liang, P.: Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190 (2021)
Original
2021
Earlier work this paper cites.

Similar

Liu, B., Zhan, L.M., Xu, et al.: Slake: A semantically-labeled knowledge-enhanced dataset for medical visual question answering. In: 2021 ISBI. pp. 1650–1654 (2021)
2021
Cited alongside, same era.
Liu, X., Ji, K., Fu, Y., Tam, et al.: P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. arXiv preprint arXiv:2110.07602 (2021)
Original
2021
Cited alongside, same era.
Radford, A., Kim, J.W., Hallacy, et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763. PMLR (2021)
2021
Cited alongside, same era.
Wei, J., Bosma, M., Zhao, V.Y., Guu, et al.: Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652 (2021)
Original
2021
Cited alongside, same era.

Then

Li, P., Liu, G., He, et al.: Masked vision and language pre-training with unimodal and multimodal contrastive losses for medical visual question answering. In: MICCAI. pp. 374–383. Springer (2023)
2023
Later among the works it cites.
Liu, H., Li, C., Wu, et al.: Visual instruction tuning. arXiv preprint arXiv:2304.08485 (2023)
Original
2023
Later among the works it cites.
Liu, X., Zheng, Y., Du, Z., Ding, et al.: Gpt understands, too. AI Open (2023)
2023
Later among the works it cites.
Zhang, R., Han, J., Zhou, A., Hu, X., Yan, S., Lu, P., Li, H., Gao, P., Qiao, Y.: Llama-adapter: Efficient fine-tuning of language models with zero-init attention. arXiv preprint arXiv:2303.16199 (2023)
Original
2023
Later among the works it cites.
Zhu, D., Chen, J., Shen, X., Li, X., Elhoseiny, M.: Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592 (2023)

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2024

Can LLMs' Tuning Methods Work in Medical Multimodal Domain?

Chen, Jiawei, Jiang, Yue, Yang, Dingkang et al.

Understand

While Large Language Models (LLMs) excel in world knowledge understanding, adapting them to specific subfields requires precise adjustments.

Due to the model's vast scale, traditional global fine-tuning methods for large models can be computationally expensive and impact generalization.
To address this challenge, a range of innovative Parameters-Efficient Fine-Tuning (PEFT) methods have emerged and achieved remarkable success in both LLMs and Large Vision-Language Models (LVLMs).
In the medical domain, fine-tuning a medical Vision-Language Pretrained (VLP) model is essential for adapting it to specific tasks.

Built on

Vaswani, A., Shazeer, N., Parmar, et al.: Attention is all you need. NIPS 30
2017
Earlier work this paper cites.
Devlin, J., Chang, M.W., Lee, et al.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Original
2018
Earlier work this paper cites.
Lau, J.J., Gayen, et al.: A dataset of clinically generated visual questions and answers about radiology images. Scientific data 5
2018
Earlier work this paper cites.
Houlsby, N., Giurgiu, A., Jastrzebski, et al.: Parameter-efficient transfer learning for nlp. In: International Conference on Machine Learning. pp. 2790–2799. PMLR (2019)
2019
Earlier work this paper cites.
Brown, T., Mann, B., Ryder, et al.: Language models are few-shot learners. Advances in neural information processing systems 33
2020
Earlier work this paper cites.
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)
Original
2020
Earlier work this paper cites.
Hu, E.J., Shen, et al.: Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021)
Original
2021
Earlier work this paper cites.
Lester, B., Al-Rfou, R., Constant, N.: The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691 (2021)
Original
2021
Earlier work this paper cites.
Li, X.L., Liang, P.: Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190 (2021)
Original
2021
Earlier work this paper cites.

Similar

Liu, B., Zhan, L.M., Xu, et al.: Slake: A semantically-labeled knowledge-enhanced dataset for medical visual question answering. In: 2021 ISBI. pp. 1650–1654 (2021)
2021
Cited alongside, same era.
Liu, X., Ji, K., Fu, Y., Tam, et al.: P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. arXiv preprint arXiv:2110.07602 (2021)
Original
2021
Cited alongside, same era.
Radford, A., Kim, J.W., Hallacy, et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763. PMLR (2021)
2021
Cited alongside, same era.
Wei, J., Bosma, M., Zhao, V.Y., Guu, et al.: Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652 (2021)
Original
2021
Cited alongside, same era.

Then

Li, P., Liu, G., He, et al.: Masked vision and language pre-training with unimodal and multimodal contrastive losses for medical visual question answering. In: MICCAI. pp. 374–383. Springer (2023)
2023
Later among the works it cites.
Liu, H., Li, C., Wu, et al.: Visual instruction tuning. arXiv preprint arXiv:2304.08485 (2023)
Original
2023
Later among the works it cites.
Liu, X., Zheng, Y., Du, Z., Ding, et al.: Gpt understands, too. AI Open (2023)
2023
Later among the works it cites.
Zhang, R., Han, J., Zhou, A., Hu, X., Yan, S., Lu, P., Li, H., Gao, P., Qiao, Y.: Llama-adapter: Efficient fine-tuning of language models with zero-init attention. arXiv preprint arXiv:2303.16199 (2023)
Original
2023
Later among the works it cites.
Zhu, D., Chen, J., Shen, X., Li, X., Elhoseiny, M.: Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592 (2023)

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Chen, Z., Du, Y., Hu, et al.: Multi-modal masked autoencoders for medical vision-and-language pre-training. In: MICCAI. pp. 679–689. Springer (2022)

2022

Cited alongside, same era.

Cong, F., Xu, S., et al.: Caption-aware medical vqa via semantic focusing and progressive cross-modality comprehension. In: ACM MM. pp. 3569–3577 (2022)

2022

Cited alongside, same era.

Li, J., Li, D., Xiong, et al.: Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In: ICCV. pp. 12888–12900 (2022)

2022

Cited alongside, same era.

Liu, H., Tam, D., Muqeeth, et al.: Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. Advances in Neural Information Processing Systems 35

2022

Cited alongside, same era.

Li, C., Wong, C., Zhang, et al.: Llava-med: Training a large language-and-vision assistant for biomedicine in one day. arXiv preprint arXiv:2306.00890 (2023)

Original

2023

Cited alongside, same era.

Original

2023

Later among the works it cites.

Chen, J., Yang, D., Jiang, Y., Li, M., Wei, J., Hou, X., Zhang, L.: Efficiency in focus: Layernorm as a catalyst for fine-tuning medical visual language pre-trained models. arXiv preprint arXiv:2404.16385 (2024)

Original

2024