Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Models

Understand

Video-based large language models (Video-LLMs) have been recently introduced, targeting both fundamental improvements in perception and comprehension, and a diverse range of user inquiries.

• In pursuit of the ultimate goal of achieving artificial general intelligence, a truly intelligent Video-LLM model should not only see and understand the surroundings, but also possess human-level commonsense, and make well-informed decisions for the users.
• To guide the development of such a model, the establishment of a robust and comprehensive evaluation system becomes crucial.
• To this end, this paper proposes \textit{Video-Bench}, a new comprehensive benchmark along with a toolkit specifically designed for evaluating Video-LLMs.