II-Bench: An Image Implication Understanding Benchmark for Multimodal Large Language Models

Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P. and Zitnick, C. L. [2014], Microsoft coco: Common objects in context

2014

Earlier work this paper cites.

Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L. and Parikh, D. [2015], Vqa: Visual question answering

2015

Earlier work this paper cites.

Plummer, B. A., Wang, L., Cervantes, C. M., Caicedo, J. C., Hockenmaier, J. and Lazebnik, S. [2015], Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models

2015

Earlier work this paper cites.

Goyal, Y., Khot, T., Summers-Stay, D., Batra, D. and Parikh, D. [2017], Making the v in vqa matter: Elevating the role of image understanding in visual question answering

2017

Earlier work this paper cites.

Kafle, K. and Kanan, C. [2017], An analysis of visual question answering algorithms

2017

Earlier work this paper cites.

Agrawal, H., Desai, K., Wang, Y., Chen, X., Jain, R., Johnson, M., Batra, D., Parikh, D., Lee, S. and Anderson, P. [2019], Nocaps: Novel object captioning at scale

2019

Earlier work this paper cites.

Hossain, M. Z., Sohel, F., Shiratuddin, M. F. and Laga, H. [2019], ‘A comprehensive survey of deep learning for image captioning’, ACM Computing Surveys (CsUR)

2019

Earlier work this paper cites.

Hudson, D. A. and Manning, C. D. [2019], Gqa: A new dataset for real-world visual reasoning and compositional question answering

2019

Earlier work this paper cites.

Singh, A., Natarjan, V., Shah, M., Jiang, Y., Chen, X., Parikh, D. and Rohrbach, M. [2019], Towards vqa models that can read

2019

Earlier work this paper cites.

Zellers, R., Holtzman, A., Bisk, Y., Farhadi, A. and Choi, Y. [2019], ‘Hellaswag: Can a machine really finish your sentence?’

2019

Earlier work this paper cites.

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T. et al. [2020], ‘An image is worth 16x16 words: Transformers for image recognition at scale’, ICLR

2020

Earlier work this paper cites.

Rahman, W., Hasan, M. K., Lee, S., Zadeh, A., Mao, C., Morency, L.-P. and Hoque, E. [2020], Integrating multimodal information in large pretrained transformers

2020

Earlier work this paper cites.

Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M. et al. [2021], ‘Training verifiers to solve math word problems’

2021

Earlier work this paper cites.

Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D. and Steinhardt, J. [2021], ‘Measuring massive multitask language understanding’, ICLR

2021

Earlier work this paper cites.

Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D. and Steinhardt, J. [2021], ‘Measuring mathematical problem solving with the math dataset’, NeurIPS

2021

Earlier work this paper cites.

Mathew, M., Karatzas, D. and Jawahar, C. V. [2021], ‘Docvqa: A dataset for vqa on document images’

2021

Earlier work this paper cites.

Singh, A., Natarajan, V., Shah, M., Jiang, Y., Chen, X. et al. [2021], ‘Towards vqa models that can read’

2021

Earlier work this paper cites.

Desai, P., Chakraborty, T. and Akhtar, M. S. [2022], ‘Nice perfume. how long did you marinate in it? multimodal sarcasm explanation’, AAAI

2022

Earlier work this paper cites.

Lu, P., Mishra, S., Xia, T., Qiu, L., Chang, K.-W., Zhu, S.-C., Tafjord, O., Clark, P. and Kalyan, A. [2022], Learn to explain: Multimodal reasoning via thought chains for science question answering

2022

Earlier work this paper cites.

Suzgun, M., Scales, N., Schärli, N., Gehrmann, S., Tay, Y., Chung, H. W. et al. [2022], ‘Challenging big-bench tasks and whether chain-of-thought can solve them’

2022

Earlier work this paper cites.

Bai, J., Bai, S., Yang, S., Wang, S., Tan, S., Wang, P., Lin, J., Zhou, C. and Zhou, J. [2023], ‘Qwen-vl: A frontier large vision-language model with versatile abilities’, arXiv preprint arXiv:2308.12966

Original

2023

Earlier work this paper cites.

Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E. et al. [2023], ‘Sparks of artificial general intelligence: Early experiments with gpt-4’, arXiv preprint arXiv: 2303.12712

2023

Earlier work this paper cites.

Cai, R., Song, Z., Guan, D., Chen, Z., Luo, X., Yi, C. and Kot, A. [2023], ‘Benchlmm: Benchmarking cross-style visual capability of large multimodal models’, arXiv preprint arXiv:2312.02896

Original

2023

Earlier work this paper cites.

Fu, C., Chen, P., Shen, Y., Qin, Y., Zhang, M., Lin, X., Yang, J., Zheng, X., Li, K., Sun, X. et al. [2023], ‘Mme: A comprehensive evaluation benchmark for multimodal large language models’, arXiv preprint arXiv:2306.13394

Original

2023

Cited alongside, same era.

Ghandi, T., Pourreza, H. and Mahyar, H. [2023], ‘Deep learning approaches on image captioning: A review’, ACM Computing Surveys

2023

Cited alongside, same era.

Hessel, J., Marasovic, A., Hwang, J. D., Lee, L., Da, J., Zellers, R., Mankoff, R. and Choi, Y. [2023], Do androids laugh at electric sheep? humor “understanding” benchmarks from the new yorker caption contest

2023

Cited alongside, same era.

Hu, J., Yao, Y., Wang, C., Wang, S., Pan, Y., Chen, Q., Yu, T., Wu, H., Zhao, Y. et al. [2023], ‘Large multilingual models pivot zero-shot multimodal learning across languages’, arXiv preprint arXiv:2308.12038

Original

2023

Cited alongside, same era.

Zhang, W., Aljunied, S. M., Gao, C., Chia, Y. K. and Bing, L. [2023], ‘M3exam: A multilingual, multimodal, multilevel benchmark for examining large language models’, arXiv preprint arXiv:2306.05179

Original

2023

Later among the works it cites.

Chen, L., Li, J., Dong, X., Zhang, P., Zang, Y. et al. [2024], ‘Are we on the right way for evaluating large vision-language models?’

2024

Closest in time.

Chen, Z., Wang, W., Tian, H., Ye, S., Gao, Z., Cui, E., Tong, W., Hu, K., Luo, J., Ma, Z. et al. [2024], ‘How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites’, arXiv preprint arXiv:2404.16821

Original

2024

Closest in time.

Cui, C., Ma, Y., Cao, X., Ye, W., Zhou, Y., Liang, K., Chen, J., Lu, J., Yang, Z., Liao, K.-D. et al. [2024], A survey on multimodal large language models for autonomous driving

2024

Closest in time.