VocalBench: Benchmarking the Vocal Conversational Abilities for Speech Interaction Models

Understand

Speech large language models (SpeechLLMs) have extended human-machine interactions from the text modality to the dynamic speech domain.

• Spoken dialogues convey diverse information, including semantic concepts, acoustic variations, paralanguage cues, and environmental context.
• However, existing evaluations of speech interaction models lack instances mimicking real scenarios and predominantly focus on the performance of distinct aspects, lacking a comprehensive comparison of critical capabilities between current routines.
• To address this gap, we propose VocalBench to assess the speech conversational abilities, comprising around 24k carefully curated instances of both English and Mandarin across four key dimensions - semantic quality, acoustic performance, conversational abilities, and robustness, covering 14 user-oriented characters.