From Pixels to Tokens: Revisiting Object Hallucinations in Large Vision-Language Models

Sgdr: Stochastic gradient descent with warm restarts

Original

Ilya Loshchilov and Frank Hutter. 2016 · 2016

Earlier work this paper cites.

An image is worth 16x16 words: Transformers for image recognition at scale

Original

Alexey Dosovitskiy. 2020 · 2020

Earlier work this paper cites.

A survey on instance segmentation: state of the art

Abdul Mueed Hafiz and Ghulam Mohiuddin Bhat. 2020 · 2020

Earlier work this paper cites.

Cascade R-CNN: High Quality Object Detection and Instance Segmentation

Zhaowei Cai and Nuno Vasconcelos. 2021 · 2021

Earlier work this paper cites.

TransVG: End-to-End Visual Grounding With Transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) . 1769–1779

Jiajun Deng, Zhengyuan Yang, Tianlang Chen, Wengang Zhou, and Houqiang Li. 2021 · 2021

Earlier work this paper cites.

Learning transferable visual models from natural language supervision. In International conference on machine learning . PmLR, 8748–8763

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al · 2021

Earlier work this paper cites.

Mlp-mixer: An all-mlp architecture for vision

Ilya O Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, et al · 2021

Earlier work this paper cites.

Flamingo: a visual language model for few-shot learning

Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al · 2022

Earlier work this paper cites.

From show to tell: A survey on deep learning-based image captioning

Matteo Stefanini, Marcella Cornia, Lorenzo Baraldi, Silvia Cascianelli, Giuseppe Fiameni, and Rita Cucchiara. 2022 · 2022

Earlier work this paper cites.

Minigpt-v2: large language model as a unified interface for vision-language multi-task learning

Original

Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechun Liu, Pengchuan Zhang, Raghuraman Krishnamoorthi, Vikas Chandra, Yunyang Xiong, and Mohamed Elhoseiny. 2023b · 2023

Earlier work this paper cites.

Sharegpt4v: Improving large multi-modal models with better captions

Original

Lin Chen, Jisong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. 2023a · 2023

Earlier work this paper cites.

InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning. In Advances in Neural Information Processing Systems , Vol. 36. 49250–49267

Wenliang Dai, Junnan Li, DONGXU LI, Anthony Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale N Fung, and Steven Hoi. 2023 · 2023

Earlier work this paper cites.

Eva: Exploring the limits of masked visual representation learning at scale. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 19358–19369

Yuxin Fang, Wen Wang, Binhui Xie, Quan Sun, Ledell Wu, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. 2023 · 2023

Earlier work this paper cites.

MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models

Original

Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, et al · 2023

Earlier work this paper cites.

Multimodal-gpt: A vision and language model for dialogue with humans

Original

Tao Gong, Chengqi Lyu, Shilong Zhang, Yudong Wang, Miao Zheng, Qian Zhao, Kuikun Liu, Wenwei Zhang, Ping Luo, and Kai Chen. 2023 · 2023

Earlier work this paper cites.

Evaluating Object Hallucination in Large Vision-Language Models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing . 292–305

Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. 2023a · 2023

Cited alongside, same era.

Eva-clip: Improved training techniques for clip at scale

Original

Quan Sun, Yuxin Fang, Ledell Wu, Xinlong Wang, and Yue Cao. 2023a · 2023

Cited alongside, same era.

Aligning large multimodal models with factually augmented rlhf

Original

Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang-Yan Gui, Yu-Xiong Wang, Yiming Yang, et al · 2023

Cited alongside, same era.

Llama: Open and efficient foundation language models

Original

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al · 2023

Cited alongside, same era.

Fgaif: Aligning large vision-language models with fine-grained ai feedback

Original

Liqiang Jing and Xinya Du. 2024 · 2024

Closest in time.

How to Configure Good In-Context Sequence for Visual Question Answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 26710–26720

Li Li, Jiawei Peng, Huiyi Chen, Chongyang Gao, and Xu Yang. 2024 · 2024

Closest in time.

Visual instruction tuning

Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2024d · 2024

Closest in time.

A survey on hallucination in large vision-language models

Original

Hanchao Liu, Wenyuan Xue, Yifei Chen, Dapeng Chen, Xiutian Zhao, Ke Wang, Liping Hou, Rongjun Li, and Wei Peng. 2024e · 2024

Closest in time.

Phd: A prompted visual hallucination evaluation dataset

Original