ViDoRe Benchmark V2: Raising the Bar for Visual Retrieval

Understand

The ViDoRe Benchmark V1 was approaching saturation with top models exceeding 90% nDCG@5, limiting its ability to discern improvements.

• ViDoRe Benchmark V2 introduces realistic, challenging retrieval scenarios via blind contextual querying, long and cross-document queries, and a hybrid synthetic and human-in-the-loop query generation process.
• It comprises four diverse, multilingual datasets and provides clear evaluation instructions.
• Initial results demonstrate substantial room for advancement and highlight insights on model generalization and multilingual capability.