NTSEBENCH: Cognitive Reasoning Benchmark for Vision Language Models

Understand

Cognitive textual and visual reasoning tasks, including puzzles, series, and analogies, demand the ability to quickly reason, decipher, and evaluate patterns both textually and spatially.

• Due to extensive training on vast amounts of human-curated data, LLMs and VLMs excel in common-sense reasoning tasks, however still struggle with more complex reasoning that demands deeper cognitive understanding.
• We introduce NTSEBench, a new dataset designed to evaluate cognitive multi-modal reasoning and problem-solving skills of large models.
• The dataset contains 2728 multiple-choice questions, accompanied by a total of 4,642 images, categorized into 26 different types.