An Exploratory Study of AI System Risk Assessment from the Lens of Data Distribution and Uncertainty

Understand

Deep learning (DL) has become a driving force and has been widely adopted in many domains and applications with competitive performance.

• In practice, to solve the nontrivial and complicated tasks in real-world applications, DL is often not used standalone, but instead contributes as a piece of gadget of a larger complex AI system.
• Although there comes a fast increasing trend to study the quality issues of deep neural networks (DNNs) at the model level, few studies have been performed to investigate the quality of DNNs at both the unit level and the potential impacts on the system level.
• More importantly, it also lacks systematic investigation on how to perform the risk assessment for AI systems from unit level to system level.