Fetching the paper…
Reading the bibliography…
Current open-source Large Multimodal Models (LMMs) excel at tasks such as open-vocabulary language grounding and segmentation but can suffer under false premises when queries imply the existence of something that is not actually present in the image.
Im2text: Describing images using 1 million captioned photographs
Vicente Ordonez, Girish Kulkarni, and Tamara Berg · 2011
Earlier work this paper cites.
Automatic caption generation for news images
Yansong Feng and Mirella Lapata · 2012
Earlier work this paper cites.
From captions to visual concepts and back
Hao Fang, Saurabh Gupta, Forrest Iandola, Rupesh K Srivastava, Li Deng, Piotr Dollár, Jianfeng Gao, Xiaodong He, Margaret Mitchell, John C Platt, et al · 2015
Earlier work this paper cites.
Deep visual-semantic alignments for generating image descriptions
Andrej Karpathy and Li Fei-Fei · 2015
Earlier work this paper cites.
Parsenet: Looking wider to see better
Wei Liu, Andrew Rabinovich, and Alexander C. Berg · 2015
Earlier work this paper cites.
Learning deconvolution network for semantic segmentation
Hyeonwoo Noh, Seunghoon Hong, and Bohyung Han · 2015
Earlier work this paper cites.
U-net: Convolutional networks for biomedical image segmentation
Olaf Ronneberger, Philipp Fischer, and Thomas Brox · 2015
Earlier work this paper cites.
Show, attend and tell: Neural image caption generation with visual attention
Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio · 2015
Earlier work this paper cites.
Segmentation from natural language expressions
Ronghang Hu, Marcus Rohrbach, and Trevor Darrell · 2016
Earlier work this paper cites.
Generation and comprehension of unambiguous object descriptions
Junhua Mao, Jonathan Huang, Alexander Toshev, Oana Camburu, Alan L Yuille, and Kevin Murphy · 2016
Earlier work this paper cites.
Question relevance in vqa: Identifying non-visual and false-premise questions
Arijit Ray, Gordon Christie, Mohit Bansal, Dhruv Batra, and Devi Parikh · 2016
Earlier work this paper cites.
Multi-scale context aggregation by dilated convolutions
Fisher Yu and Vladlen Koltun · 2016
Earlier work this paper cites.
Modeling context in referring expressions
Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara L Berg · 2016
Earlier work this paper cites.
Segnet: A deep convolutional encoder-decoder architecture for image segmentation
Vijay Badrinarayanan, Alex Kendall, and Roberto Cipolla · 2017
Earlier work this paper cites.
Mask r-cnn
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick · 2017
Earlier work this paper cites.
Clevr: A diagnostic dataset for compositional language and elementary visual reasoning
Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick · 2017
Earlier work this paper cites.
The promise of premise: Harnessing question premises in visual question answering
Aroma Mahendru, Viraj Prabhu, Akrit Mohapatra, Dhruv Batra, and Stefan Lee · 2017
Earlier work this paper cites.
Image-grounded conversations: Multimodal context for natural question and response generation
Nasrin Mostafazadeh, Chris Brockett, Bill Dolan, Michel Galley, Jianfeng Gao, Georgios P Spithourakis, and Lucy Vanderwende · 2017
Earlier work this paper cites.
Fully convolutional networks for semantic segmentation
Evan Shelhamer, Jonathan Long, and Trevor Darrell · 2017
Earlier work this paper cites.
A corpus of natural language for visual reasoning
Alane Suhr, Mike Lewis, James Yeh, and Yoav Artzi · 2017
Earlier work this paper cites.
Pyramid scene parsing network
Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia · 2017
Earlier work this paper cites.
Scene parsing through ade20k dataset
Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba · 2017
Earlier work this paper cites.
Coco-stuff: Thing and stuff classes in context
Holger Caesar, Jasper Uijlings, and Vittorio Ferrari · 2018
Earlier work this paper cites.
Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs
Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L. Yuille · 2018
Earlier work this paper cites.
Learning to evaluate image captioning
Yin Cui, Guandao Yang, Andreas Veit, Xun Huang, and Serge J. Belongie · 2018
Earlier work this paper cites.
Vizwiz grand challenge: Answering visual questions from blind people
Danna Gurari, Qing Li, Abigale J Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey P Bigham · 2018
Earlier work this paper cites.
Question relevance in visual question answering
Prakruthi Prabhakar, Nitish Kulkarni, and Linghao Zhang · 2018
Earlier work this paper cites.
Object hallucination in image captioning
Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko · 2018
Cited alongside, same era.
Denseaspp for semantic segmentation in street scenes
Maoke Yang, Kun Yu, Chi Zhang, Zhiwei Li, and Kuiyuan Yang · 2018
Cited alongside, same era.
Dual attention network for scene segmentation
Jun Fu, Jing Liu, Haijie Tian, Yong Li, Yongjun Bao, Zhiwei Fang, and Hanqing Lu · 2019
Cited alongside, same era.
Ccnet: Criss-cross attention for semantic segmentation
Zilong Huang, Xinggang Wang, Lichao Huang, Chang Huang, Yunchao Wei, and Wenyu Liu · 2019
Cited alongside, same era.
Gqa: A new dataset for real-world visual reasoning and compositional question answering
Drew A Hudson and Christopher D Manning · 2019
Cited alongside, same era.
TIGEr: Text-to-image grounding for image caption evaluation
K-net: Towards unified image segmentation
Wenwei Zhang, Jiangmiao Pang, Kai Chen, and Chen Change Loy · 2021
Later among the works it cites.
Masked-attention mask transformer for universal image segmentation
Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexander Kirillov, and Rohit Girdhar · 2022
Later among the works it cites.
Vlt: Vision-language transformer and query generation for referring segmentation
Henghui Ding, Chang Liu, Suchen Wang, and Xudong Jiang · 2022
Later among the works it cites.
Deconfounded visual grounding
Jianqiang Huang, Yu Qin, Jiaxin Qi, Qianru Sun, and Hanwang Zhang · 2022
Later among the works it cites.
Semantic multi-modal reprojection for robust visual question answering
Akib Mashrur, Wei Luo, Nayyar A Zaidi, and Antonio Robles-Kelly · 2022
Later among the works it cites.
Adaptive perspective distillation for semantic segmentation
Zhuotao Tian, Pengguang Chen, Xin Lai, Li Jiang, Shu Liu, Hengshuang Zhao, Bei Yu, Ming-Chang Yang, and Jiaya Jia · 2022
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Ming Jiang, Qiuyuan Huang, Lei Zhang, Xin Wang, Pengchuan Zhang, Zhe Gan, Jana Diesner, and Jianfeng Gao · 2019
Cited alongside, same era.
Panoptic segmentation
Alexander Kirillov, Kaiming He, Ross Girshick, Carsten Rother, and Piotr Dollár · 2019
Cited alongside, same era.
Clevr-ref+: Diagnosing visual reasoning with referring expressions
Runtao Liu, Chenxi Liu, Yutong Bai, and Alan L Yuille · 2019
Cited alongside, same era.
Question action relevance and editing for visual question answering
Andeep S Toor, Harry Wechsler, and Michele Nappi · 2019
Cited alongside, same era.
Upsnet: A unified panoptic segmentation network
Yuwen Xiong, Renjie Liao, Hengshuang Zhao, Rui Hu, Min Bai, Ersin Yumer, and Raquel Urtasun · 2019
Cited alongside, same era.
Asymmetric non-local neural networks for semantic segmentation
Zhen Zhu, Mengde Xu, Song Bai, Tengteng Huang, and Xiang Bai · 2019
Cited alongside, same era.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 2020
Cited alongside, same era.
Later among the works it cites.
Cris: Clip-driven referring image segmentation
Zhaoqing Wang, Yu Lu, Qiang Li, Xunqiang Tao, Yandong Guo, Mingming Gong, and Tongliang Liu · 2022
Later among the works it cites.
Chain-of-thought prompting elicits reasoning in large language models
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al · 2022
Later among the works it cites.
Reliable visual question answering: Abstain rather than answer incorrectly
Spencer Whitehead, Suzanne Petryk, Vedaad Shakib, Joseph Gonzalez, Trevor Darrell, Anna Rohrbach, and Marcus Rohrbach · 2022
Later among the works it cites.
Towards robust referring image segmentation
Jianzong Wu, Xiangtai Li, Xia Li, Henghui Ding, Yunhai Tong, and Dacheng Tao · 2022
Later among the works it cites.
Lavt: Language-aware vision transformer for referring image segmentation
Zhao Yang, Jiaqi Wang, Yansong Tang, Kai Chen, Hengshuang Zhao, and Philip HS Torr · 2022
Later among the works it cites.
Clair: Evaluating image captions with large language models
David Chan, Suzanne Petryk, Joseph E Gonzalez, Trevor Darrell, and John Canny · 2023
Closest in time.
Visual programming for text-to-image generation and evaluation
Jaemin Cho, Abhay Zala, and Mohit Bansal · 2023
Closest in time.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al · 2023
Closest in time.
Lisa: Reasoning segmentation via large language model
Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia · 2023
Closest in time.
Robust visual question answering via semantic cross modal augmentation
Akib Mashrur, Wei Luo, Nayyar A Zaidi, and Antonio Robles-Kelly · 2023
Closest in time.
OpenAI · 2023
Closest in time.
Gpt-4v(ision) system card, 2023
OpenAI · 2023
Closest in time.
Kosmos-2: Grounding multimodal large language models to the world
Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei · 2023
Closest in time.
Paco: Parts and attributes of common objects
Vignesh Ramanathan, Anmol Kalia, Vladan Petrovic, Yi Wen, Baixue Zheng, Baishan Guo, Rui Wang, Aaron Marquez, Rama Kovvuri, Abhishek Kadian, et al · 2023
Closest in time.
Contrastive grouping with transformer for referring image segmentation
Jiajin Tang, Ge Zheng, Cheng Shi, and Sibei Yang · 2023
Closest in time.
Learning context-aware classifier for semantic segmentation
Zhuotao Tian, Jiequan Cui, Li Jiang, Xiaojuan Qi, Xin Lai, Yixin Chen, Shu Liu, and Jiaya Jia · 2023
Closest in time.
Ferret: Refer and ground anything anywhere at any granularity, 2023
Haoxuan You, Haotian Zhang, Zhe Gan, Xianzhi Du, Bowen Zhang, Zirui Wang, Liangliang Cao, Shih-Fu Chang, and Yinfei Yang · 2023
Closest in time.
Contextual object detection with multimodal large language models
Yuhang Zang, Wei Li, Jun Han, Kaiyang Zhou, and Chen Change Loy · 2023
Closest in time.
Gpt4roi: Instruction tuning large language model on region-of-interest
Shilong Zhang, Peize Sun, Shoufa Chen, Min Xiao, Wenqi Shao, Wenwei Zhang, Kai Chen, and Ping Luo · 2023
Closest in time.
Bubogpt: Enabling visual grounding in multi-modal llms
Yang Zhao, Zhijie Lin, Daquan Zhou, Zilong Huang, Jiashi Feng, and Bingyi Kang · 2023
Closest in time.