Fetching the paper…

Reading the bibliography…

2024

Hallucination Benchmark in Medical Visual Question Answering

Wu, Jinge, Kim, Yunsoo, Wu, Honghan

Original
PDF
Post on X

Understand

The recent success of large language and vision models (LLVMs) on vision question answering (VQA), particularly their applications in medicine (Med-VQA), has shown a great potential of realizing effective visual assistants for healthcare.

However, these models are not extensively tested on the hallucination phenomenon in clinical settings.
Here, we created a hallucination benchmark of medical images paired with question-answer sets and conducted a comprehensive evaluation of the state-of-the-art models.
The study provides an in-depth analysis of current models' limitations and reveals the effectiveness of various prompting strategies.

Hallucination Benchmark in Medical Visual Question Answering · Around

Built on

A dataset of clinically generated visual questions and answers about radiology images
Jason J Lau, Soumya Gayen, Asma Ben Abacha, and Dina Demner-Fushman · 2018
Earlier work this paper cites.
Pathological visual question answering
Original
Xuehai He, Zhuo Cai, Wenlan Wei, Yichen Zhang, Luntian Mou, Eric Xing, and Pengtao Xie · 2020
Earlier work this paper cites.

Similar

Llava-med: Training a large language-and-vision assistant for biomedicine in one day
Original
Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao · 2023
Cited alongside, same era.
Med-halt: Medical domain hallucination test for large language models
Original
Logesh Kumar Umapathi, Ankit Pal, and Malaikannan Sankarasubbu · 2023
Cited alongside, same era.
Improved baselines with visual instruction tuning
Original
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee
Cited in the paper.
Visual instruction tuning
Original
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee
Cited in the paper.

Then

Pmc-vqa: Visual instruction tuning for medical visual question answering
Original
Xiaoman Zhang, Chaoyi Wu, Ziheng Zhao, Weixiong Lin, Ya Zhang, Yanfeng Wang, and Weidi Xie · 2023
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…