iVISPAR -- An Interactive Visual-Spatial Reasoning Benchmark for VLMs

A formal basis for the heuristic determination of minimum cost paths

Peter E Hart, Nils J Nilsson, and Bertram Raphael. 1968 · 1968

Earlier work this paper cites.

Probing contextual language models for common ground with visual representations

Original

Gabriel Ilharco, Rowan Zellers, Ali Farhadi, and Hannaneh Hajishirzi. 2021 · 2005

Earlier work this paper cites.

An image is worth 16x16 words: Transformers for image recognition at scale

Original

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2020 · 2010

Earlier work this paper cites.

CLEVR: A diagnostic dataset for compositional language and elementary visual reasoning

Original

Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Li Fei-Fei, C. Lawrence Zitnick, and Ross Girshick. 2016 · 2016

Earlier work this paper cites.

Learning language games through interaction

Original

Sida I. Wang, Percy Liang, and Christopher D. Manning. 2016 · 2016

Earlier work this paper cites.

ShapeWorld - a new test methodology for multimodal language understanding

Original

Alexander Kuhnle and Ann Copestake. 2017 · 2017

Earlier work this paper cites.

Can language models encode perceptual structure without grounding? a case study in color

Original

Mostafa Abdou, Artur Kulmizev, Daniel Hershcovich, Stella Frank, Ellie Pavlick, and Anders Søgaard. 2021 · 2021

Earlier work this paper cites.

SpartQA: : A textual question answering benchmark for spatial reasoning

Original

Roshanak Mirzaee, Hossein Rajaby Faghihi, Qiang Ning, and Parisa Kordjmashidi. 2021 · 2021

Earlier work this paper cites.

Mapping language models to grounded conceptual spaces

Roma Patel and Ellie Pavlick. 2021 · 2021

Earlier work this paper cites.

Chain-of-thought prompting elicits reasoning in large language models

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022 · 2022

Earlier work this paper cites.

Look before you leap: Unveiling the power of GPT-4v in robotic vision-language planning

Original

Yingdong Hu, Fanqi Lin, Tong Zhang, Li Yi, and Yang Gao. 2023 · 2023

Earlier work this paper cites.

What’s "up" with vision-language models? investigating their struggle with spatial reasoning

Amita Kamath, Jack Hessel, and Kai-Wei Chang. 2023a · 2023

Earlier work this paper cites.

Super-CLEVR: A virtual benchmark to diagnose domain robustness in visual reasoning

Original

Zhuowan Li, Xingrui Wang, Elias Stengel-Eskin, Adam Kortylewski, Wufei Ma, Benjamin Van Durme, and Alan Yuille. 2023 · 2023

Earlier work this paper cites.

Visual spatial reasoning

Original

Fangyu Liu, Guy Emerson, and Nigel Collier. 2023 · 2023

Earlier work this paper cites.

Linearly mapping from image to text space

Original

Jack Merullo, Louis Castricato, Carsten Eickhoff, and Ellie Pavlick. 2023 · 2023

Earlier work this paper cites.

3d-aware visual question answering about parts, poses and occlusions

Xingrui Wang, Wufei Ma, Zhuowan Li, Adam Kortylewski, and Alan Yuille. 2023 · 2023

Earlier work this paper cites.

The rise and potential of large language model based agents: A survey

Original

Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, and 1 others. 2023 · 2023

Cited alongside, same era.

Large language models for robotics: A survey

Original

Fanlong Zeng, Wensheng Gan, Yongheng Wang, Ning Liu, and Philip S Yu. 2023 · 2023

Cited alongside, same era.

Sigmoid loss for language image pre-training

Original

Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023 · 2023

Cited alongside, same era.

Can large language models be good path planners? a benchmark and investigation on spatial-temporal reasoning

Original

Mohamed Aghzal, Erion Plaku, and Ziyu Yao. 2024 · 2024

Cited alongside, same era.

A little less conversation, a little more action, please: Investigating the physical common-sense of LLMs in a 3d embodied environment

Original

Matteo G. Mecattaf, Ben Slater, Marko Tešić, Jonathan Prunty, Konstantinos Voudouris, and Lucy G. Cheke. 2024 · 2024

Later among the works it cites.

Sliding puzzles gym: A scalable benchmark for state representation in visual reinforcement learning

Bryan Lincoln Marques de Oliveira, Bruno Brandão, Murilo Lopes da Luz, Luana Guedes Barros Martins, Telma Woerle de Lima Soares, and Luckeciano Carvalho Melo. 2024 · 2024

Later among the works it cites.

Gpt-4o system card

Original

OpenAI, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander Mądry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, Alex Nichol, and 400 others. 2024 · 2024

Later among the works it cites.

SpaRC and SpaRP: Spatial reasoning characterization and path generation for understanding spatial reasoning capability of large language models

Original

Md Imbesat Hassan Rizvi, Xiaodan Zhu, and Iryna Gurevych. 2024 · 2024