Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models

Backpropagation applied to handwritten zip code recognition

LeCun, Y.; Boser, B.; Denker, J. S.; Henderson, D.; Howard, R. E.; Hubbard, W.; and Jackel, L. D. 1989 · 1989

Earlier work this paper cites.

Deep residual learning for image recognition

He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016 · 2016

Earlier work this paper cites.

A Diagram is Worth a Dozen Images

Kembhavi, A.; Salvato, M.; Kolve, E.; Seo, M.; Hajishirzi, H.; and Farhadi, A. 2016 · 2016

Earlier work this paper cites.

Div8k: Diverse 8k resolution image dataset

Gu, S.; Lugmayr, A.; Danelljan, M.; Fritsche, M.; Lamour, J.; and Timofte, R. 2019 · 2019

Earlier work this paper cites.

OCR-VQA: Visual Question Answering by Reading Text in Images

Mishra, A.; Shekhar, S.; Singh, A. K.; and Chakraborty, A. 2019 · 2019

Earlier work this paper cites.

Towards VQA Models That Can Read

Singh, A.; Natarjan, V.; Shah, M.; Jiang, Y.; Chen, X.; Parikh, D.; and Rohrbach, M. 2019 · 2019

Earlier work this paper cites.

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; Uszkoreit, J.; and Houlsby, N. 2021 · 2021

Earlier work this paper cites.

Docvqa: A dataset for vqa on document images

Mathew, M.; Karatzas, D.; and Jawahar, C. 2021 · 2021

Earlier work this paper cites.

Learning transferable visual models from natural language supervision

Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021 · 2021

Earlier work this paper cites.

GLM: General Language Model Pretraining with Autoregressive Blank Infilling

Du, Z.; Qian, Y.; Liu, X.; Ding, M.; Qiu, J.; Yang, Z.; and Tang, J. 2022 · 2022

Earlier work this paper cites.

Unsupervised Dense Information Retrieval with Contrastive Learning

Izacard, G.; Caron, M.; Hosseini, L.; Riedel, S.; Bojanowski, P.; Joulin, A.; and Grave, E. 2022 · 2022

Earlier work this paper cites.

A convnet for the 2020s

Liu, Z.; Mao, H.; Wu, C.-Y.; Feichtenhofer, C.; Darrell, T.; and Xie, S. 2022 · 2022

Earlier work this paper cites.

Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering

Lu, P.; Mishra, S.; Xia, T.; Qiu, L.; Chang, K.-W.; Zhu, S.-C.; Tafjord, O.; Clark, P.; and Kalyan, A. 2022 · 2022

Earlier work this paper cites.

ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning

Masry, A.; Long, D.; Tan, J. Q.; Joty, S.; and Hoque, E. 2022 · 2022

Earlier work this paper cites.

Gpt-4 technical report

Achiam, J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F. L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al. 2023 · 2023

Earlier work this paper cites.

Introducing our Multimodal Models

Bavishi, R.; Elsen, E.; Hawthorne, C.; Nye, M.; Odena, A.; Somani, A.; and Taşırlar, S. 2023 · 2023

Earlier work this paper cites.

Token Merging: Your ViT but Faster

Bolya, D.; Fu, C.-Y.; Dai, X.; Zhang, P.; Feichtenhofer, C.; and Hoffman, J. 2023 · 2023

Cited alongside, same era.

MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models

Fu, C.; Chen, P.; Shen, Y.; Qin, Y.; Zhang, M.; Lin, X.; Yang, J.; Zheng, X.; Li, K.; Sun, X.; et al. 2023 · 2023

Cited alongside, same era.

CORE-MM: Complex Open-Ended Reasoning Evaluation For Multi-Modal Large Language Models

Han, X.; You, Q.; Liu, Y.; Chen, W.; Zheng, H.; Mrini, K.; Lin, X.; Wang, Y.; Zhai, B.; Yuan, J.; Wang, H.; and Yang, H. 2023 · 2023

Cited alongside, same era.

Segment anything

Kirillov, A.; Mintun, E.; Ravi, N.; Mao, H.; Rolland, C.; Gustafson, L.; Xiao, T.; Whitehead, S.; Berg, A. C.; Lo, W.-Y.; et al. 2023 · 2023

Cited alongside, same era.

OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documents

Laurençon, H.; Saulnier, L.; Tronchon, L.; Bekman, S.; Singh, A.; Lozhkov, A.; Wang, T.; Karamcheti, S.; Rush, A.; Kiela, D.; Cord, M.; and Sanh, V. 2023 · 2023

Cited alongside, same era.

VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Models

Duan, H.; Yang, J.; Qiao, Y.; Fang, X.; Chen, L.; Liu, Y.; Dong, X.; Zang, Y.; Zhang, P.; Wang, J.; et al. 2024 · 2024

Closest in time.

BLINK: Multimodal Large Language Models Can See but Not Perceive

Fu, X.; Hu, Y.; Li, B.; Feng, Y.; Wang, H.; Lin, X.; Roth, D.; Smith, N. A.; Ma, W.-C.; and Krishna, R. 2024 · 2024

Closest in time.

ConvLLaVA: Hierarchical Backbones as Visual Encoder for Large Multimodal Models

Ge, C.; Cheng, S.; Wang, Z.; Yuan, J.; Gao, Y.; Song, J.; Song, S.; Huang, G.; and Zheng, B. 2024 · 2024

Closest in time.

ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools

GLM, T.; Zeng, A.; Xu, B.; Wang, B.; Zhang, C.; Yin, D.; Rojas, D.; Feng, G.; Zhao, H.; Lai, H.; Yu, H.; Wang, H.; Sun, J.; Zhang, J.; Cheng, J.; Gui, J.; Tang, J.; Zhang, J.; Li, J.; Zhao, L.; Wu, L.; Zhong, L.; Liu, M.; Huang, M.; Zhang, P.; Zheng, Q.; Lu, R.; Duan, S.; Zhang, S.; Cao, S.; Yang, S.; Tam, W. L.; Zhao, W.; Liu, X.; Xia, X.; Zhang, X.; Gu, X.; Lv, X.; Liu, X.; Liu, X.; Yang, X.; Song, X.; Zhang, X.; An, Y.; Xu, Y.; Niu, Y.; Yang, Y.; Li, Y.; Bai, Y.; Dong, Y.; Qi, Z.; Wang, Z.; Yang, Z.; Du, Z.; Hou, Z.; and Wang, Z. 2024 · 2024

Closest in time.