Can You Explain That? Lucid Explanations Help Human-AI Collaborative Image Retrieval

Understand

While there have been many proposals on making AI algorithms explainable, few have attempted to evaluate the impact of AI-generated explanations on human performance in conducting human-AI collaborative tasks.

• To bridge the gap, we propose a Twenty-Questions style collaborative image retrieval game, Explanation-assisted Guess Which (ExAG), as a method of evaluating the efficacy of explanations (visual evidence or textual justification) in the context of Visual Question Answering (VQA).
• In our proposed ExAG, a human user needs to guess a secret image picked by the VQA agent by asking natural language questions to it.
• We show that overall, when AI explains its answers, users succeed more often in guessing the secret image correctly.