Do Prompts Really Prompt? Exploring the Prompt Understanding Capability of Whisper

Understand

This research explores how the information of prompts interacts with the high-performing speech recognition model, Whisper.

• We compare its performances when prompted by prompts with correct information and those corrupted with incorrect information.
• Our results unexpectedly show that Whisper may not understand the textual prompts in a human-expected way.
• Additionally, we find that performance improvement is not guaranteed even with stronger adherence to the topic information in textual prompts.