Combinatorial Testing for Deep Learning Systems

Understand

Deep learning (DL) has achieved remarkable progress over the past decade and been widely applied to many safety-critical applications.

• However, the robustness of DL systems recently receives great concerns, such as adversarial examples against computer vision systems, which could potentially result in severe consequences.
• Adopting testing techniques could help to evaluate the robustness of a DL system and therefore detect vulnerabilities at an early stage.
• The main challenge of testing such systems is that its runtime state space is too large: if we view each neuron as a runtime state for DL, then a DL system often contains massive states, rendering testing each state almost impossible.