Scaffolding Coordinates to Promote Vision-Language Coordination in Large Multi-Modal Models

Clevr: A diagnostic dataset for compositional language and elementary visual reasoning

Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. 2017 · 2017

Earlier work this paper cites.

Large language models are zero-shot reasoners

Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022 · 2022

Earlier work this paper cites.

Winoground: Probing vision and language models for visio-linguistic compositionality

Tristan Thrush, Ryan Jiang, Max Bartolo, Amanpreet Singh, Adina Williams, Douwe Kiela, and Candace Ross. 2022 · 2022

Earlier work this paper cites.

Chain-of-Thought prompting elicits reasoning in large language models

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022 · 2022

Earlier work this paper cites.

Gpt-4 technical report

Original

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023 · 2023

Earlier work this paper cites.

Graph of thoughts: Solving elaborate problems with large language models

Original

Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Michal Podstawski, Hubert Niewiadomski, Piotr Nyczyk, et al. 2023 · 2023

Earlier work this paper cites.

Breaking common sense: Whoops! a vision-and-language benchmark of synthetic and compositional images

Nitzan Bitton-Guetta, Yonatan Bitton, Jack Hessel, Ludwig Schmidt, Yuval Elovici, Gabriel Stanovsky, and Roy Schwartz. 2023 · 2023

Earlier work this paper cites.

Towards generic anomaly detection and understanding: Large-scale visual-linguistic model (gpt-4v) takes the lead

Original

Yunkang Cao, Xiaohao Xu, Chen Sun, Xiaonan Huang, and Weiming Shen. 2023 · 2023

Earlier work this paper cites.

Can vision-language models think from a first-person perspective?

Original

Sijie Cheng, Zhicheng Guo, Jingwen Wu, Kechen Fang, Peng Li, Huaping Liu, and Yang Liu. 2023 · 2023

Earlier work this paper cites.

Instructblip: Towards general-purpose vision-language models with instruction tuning

Original

Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. 2023 · 2023

Earlier work this paper cites.

Hallusionbench: An advanced diagnostic suite for entangled language hallucination & visual illusion in large vision-language models

Original

Tianrui Guan, Fuxiao Liu, Xiyang Wu Ruiqi Xian Zongxia Li, Xiaoyu Liu Xijun Wang, Lichang Chen Furong Huang Yaser Yacoob, and Dinesh Manocha Tianyi Zhou. 2023 · 2023

Earlier work this paper cites.

Segment anything

Original

Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. 2023 · 2023

Cited alongside, same era.

Evaluating object hallucination in large vision-language models

Original

Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. 2023 · 2023

Cited alongside, same era.

Compositional chain-of-Thought prompting for large multimodal models

Original

Chancharik Mitra, Brandon Huang, Trevor Darrell, and Roei Herzig. 2023 · 2023

Cited alongside, same era.

Gemini: a family of highly capable multimodal models

Original

Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. 2023 · 2023

Cited alongside, same era.

Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi

Original

Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. 2023 · 2023

Later among the works it cites.

Gpt4roi: Instruction tuning large language model on region-of-interest

Original

Shilong Zhang, Peize Sun, Shoufa Chen, Min Xiao, Wenqi Shao, Wenwei Zhang, Kai Chen, and Ping Luo. 2023 · 2023

Later among the works it cites.

Judging llm-as-a-judge with mt-bench and chatbot arena

Original

Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023 · 2023

Later among the works it cites.

Spatialvlm: Endowing vision-language models with spatial reasoning capabilities

Original

Boyuan Chen, Zhuo Xu, Sean Kirmani, Brian Ichter, Danny Driess, Pete Florence, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. 2024 · 2024