Visual Interpretability for Deep Learning: a Survey

Understand

This paper reviews recent studies in understanding neural-network representations and learning neural networks with interpretable/disentangled middle-layer representations.

• Although deep neural networks have exhibited superior performance in various tasks, the interpretability is always the Achilles' heel of deep neural networks.
• At present, deep neural networks obtain high discrimination power at the cost of low interpretability of their black-box representations.
• We believe that high model interpretability may help people to break several bottlenecks of deep learning, e.g., learning from very few annotations, learning via human-computer communications at the semantic level, and semantically debugging network representations.