SpokenNativQA: Multilingual Everyday Spoken Queries for LLMs

Understand

Large Language Models (LLMs) have demonstrated remarkable performance across various disciplines and tasks.

• However, benchmarking their capabilities with multilingual spoken queries remains largely unexplored.
• In this study, we introduce SpokenNativQA, the first multilingual and culturally aligned spoken question-answering (SQA) dataset designed to evaluate LLMs in real-world conversational settings.
• The dataset comprises approximately 33,000 naturally spoken questions and answers in multiple languages, including low-resource and dialect-rich languages, providing a robust benchmark for assessing LLM performance in speech-based interactions.