Beyond Logit Lens: Contextual Embeddings for Robust Hallucination Detection & Grounding in VLMs

Latent retrieval for weakly supervised open domain question answering

Original

Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. 2019 · 1906

Earlier work this paper cites.

Lxmert: Learning cross-modality encoder representations from transformers

Original

Hao Tan and Mohit Bansal. 2019 · 1908

Earlier work this paper cites.

Interpreting gpt: The logit lens

nostalgebraist. 2020 · 2020

Earlier work this paper cites.

A first look: Towards explainable textvqa models via visual and textual explanations

Original

Varun Nagaraj Rao, Xingjian Zhen, Karen Hovsepian, and Mingwei Shen. 2021 · 2021

Earlier work this paper cites.

Found a reason for me? weakly-supervised grounded visual question answering using capsules

Aisha Urooj, Hilde Kuehne, Kevin Duarte, Chuang Gan, Niels Lobo, and Mubarak Shah. 2021 · 2021

Earlier work this paper cites.

Vinvl: Revisiting visual representations in vision-language models

Pengchuan Zhang, Xiujun Li, Xiaowei Hu, Jianwei Yang, Lei Zhang, Lijuan Wang, Yejin Choi, and Jianfeng Gao. 2021 · 2021

Earlier work this paper cites.

Attributed question answering: Evaluation and modeling for attributed large language models

Original

Bernd Bohnet, Vinh Q Tran, Pat Verga, Roee Aharoni, Daniel Andor, Livio Baldini Soares, Massimiliano Ciaramita, Jacob Eisenstein, Kuzman Ganchev, Jonathan Herzig, et al. 2022 · 2022

Earlier work this paper cites.

Grounding answers for visual questions asked by visually impaired people

Chongyan Chen, Samreen Anjum, and Danna Gurari. 2022 · 2022

Earlier work this paper cites.

Rarr: Researching and revising what language models say, using language models

Original

Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Y Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, et al. 2022 · 2022

Earlier work this paper cites.

Recitation-augmented language models

Original

Zhiqing Sun, Xuezhi Wang, Yi Tay, Yiming Yang, and Denny Zhou. 2022 · 2022

Earlier work this paper cites.

The internal state of an llm knows when it’s lying

Original

Amos Azaria and Tom Mitchell. 2023 · 2023

Earlier work this paper cites.

Qwen-vl: A frontier large vision-language model with versatile abilities

Original

Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023 · 2023

Earlier work this paper cites.

Dola: Decoding by contrasting layers improves factuality in large language models

Original

Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengcheng He. 2023 · 2023

Earlier work this paper cites.

Enabling large language models to generate text with citations

Original

Tianyu Gao, Howard Yen, Jiatong Yu, and Danqi Chen. 2023 · 2023

Cited alongside, same era.

Retrieving supporting evidence for generative question answering

Siqing Huo, Negar Arabzadeh, and Charles Clarke. 2023 · 2023

Cited alongside, same era.

Towards mitigating hallucination in large language models via self-reflection

Original

Ziwei Ji, Tiezheng Yu, Yan Xu, Nayeon Lee, Etsuko Ishii, and Pascale Fung. 2023 · 2023

Cited alongside, same era.

Sentence attention blocks for answer grounding

Seyedalireza Khoshsirat and Chandra Kambhamettu. 2023 · 2023

Cited alongside, same era.

Trusting your evidence: Hallucinate less with context-aware decoding

Original

Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Scott Wen-tau Yih. 2023 · 2023

Monkey: Image resolution and text label are important things for large multi-modal models

Zhang Li, Biao Yang, Qiang Liu, Zhiyin Ma, Shuo Zhang, Jingxu Yang, Yabo Sun, Yuliang Liu, and Xiang Bai. 2024 · 2024

Closest in time.

Peering into the mind of language models: An approach for attribution in contextual question answering

Original

Anirudh Phukan, Shwetha Somasundaram, Apoorv Saxena, Koustava Goswami, and Balaji Vasan Srinivasan. 2024 · 2024

Closest in time.

Model internals-based answer attribution for trustworthy retrieval-augmented generation

Original

Jirui Qi, Gabriele Sarti, Raquel Fernández, and Arianna Bisazza. 2024 · 2024

Closest in time.

Enhancing post-hoc attributions in long document comprehension via coarse grained answer decomposition

Original

Pritika Ramu, Koustava Goswami, Apoorv Saxena, and Balaji Vasan Srinivavsan. 2024 · 2024

Closest in time.

Glamm: Pixel grounding large multimodal model